Download Full Version ebook - Visit ebookmeta.

com

Deep Learning with Swift for TensorFlow

Differentiable Programming with Swift 1st Edition
Rahul Bhalley

https://ebookmeta.com/product/deep-learning-with-swift-for-
tensorflow-differentiable-programming-with-swift-1st-
edition-rahul-bhalley/

OR CLICK HERE

DOWLOAD NOW

Discover More Ebook - Explore Now at ebookmeta.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Start reading on any device today!

Deep Learning with Swift for TensorFlow Rahul Bhalley

https://ebookmeta.com/product/deep-learning-with-swift-for-tensorflow-
rahul-bhalley/

ebookmeta.com

iOS 15 Programming Fundamentals with Swift: Swift, Xcode,

and Cocoa Basics 1st Edition Matt Neuburg

https://ebookmeta.com/product/ios-15-programming-fundamentals-with-
swift-swift-xcode-and-cocoa-basics-1st-edition-matt-neuburg/

ebookmeta.com

Combine Asynchronous Programming with Swift Third Edition

Scott Gardner

https://ebookmeta.com/product/combine-asynchronous-programming-with-
swift-third-edition-scott-gardner/

ebookmeta.com

Forced Migration and Human Security in the Eastern

Orthodox World 1st Edition Lucian N. Leustean

https://ebookmeta.com/product/forced-migration-and-human-security-in-
the-eastern-orthodox-world-1st-edition-lucian-n-leustean/

ebookmeta.com
The Everything Girl L. Maleki

https://ebookmeta.com/product/the-everything-girl-l-maleki/

ebookmeta.com

Coreference in English and German A Theoretical Framework

and Its Application in a Study of Court Decisions 1st
Edition Sanna Engell
https://ebookmeta.com/product/coreference-in-english-and-german-a-
theoretical-framework-and-its-application-in-a-study-of-court-
decisions-1st-edition-sanna-engell/
ebookmeta.com

Allied Armor in Normandy 1st Edition Yves Buffetaut

https://ebookmeta.com/product/allied-armor-in-normandy-1st-edition-
yves-buffetaut/

ebookmeta.com

A Kiss for a Kiss All in 4 1st Edition Helena Hunting

https://ebookmeta.com/product/a-kiss-for-a-kiss-all-in-4-1st-edition-
helena-hunting/

ebookmeta.com

When to Walk Away Finding Freedom from Toxic People Gary

Thomas

https://ebookmeta.com/product/when-to-walk-away-finding-freedom-from-
toxic-people-gary-thomas/

ebookmeta.com
Compressor Handbook Principles and Practice 2nd Edition
Tony Giampaolo

https://ebookmeta.com/product/compressor-handbook-principles-and-
practice-2nd-edition-tony-giampaolo/

ebookmeta.com
Deep Learning
with Swift for
TensorFlow
Differentiable Programming with Swift
—
Rahul Bhalley
 Deep Learning with
Swift for TensorFlow
Differentiable Programming
with Swift

Rahul Bhalley
Deep Learning with Swift for TensorFlow: Differentiable Programming
with Swift
Rahul Bhalley
Ludhiana, India

ISBN-13 (pbk): 978-1-4842-6329-7 ISBN-13 (electronic): 978-1-4842-6330-3

https://doi.org/10.1007/978-1-4842-6330-3

Copyright © 2021 by Rahul Bhalley

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or
part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way,
and transmission or information storage and retrieval, electronic adaptation, computer software,
or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark
symbol with every occurrence of a trademarked name, logo, or image we use the names, logos,
and images only in an editorial fashion and to the benefit of the trademark owner, with no
intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
they are not identified as such, is not to be taken as an expression of opinion as to whether or not
they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal
responsibility for any errors or omissions that may be made. The publisher makes no warranty,
express or implied, with respect to the material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Aaron Black
Development Editor: James Markham
Coordinating Editor: Jessica Vakili
Distributed to the book trade worldwide by Springer Science+Business Media New York,1 NY
Plazar, New York, NY 10014. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@
springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and
the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc).
SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail booktranslations@springernature.com; for
reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com.
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook
versions and licenses are also available for most titles. For more information, reference our Print
and eBook Bulk Sales web page at http://www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is available
to readers on GitHub via the book’s product page, located at www.apress.com/978-1-4842-6329-7.
For more detailed information, please visit http://www.apress.com/source-code.
Printed on acid-free paper
Table of Contents
About the Author���������������������������������������������������������������������������������ix

About the Technical Reviewer�������������������������������������������������������������xi

Preface����������������������������������������������������������������������������������������������xiii

Chapter 1: Machine Learning Basics�������������������������������������������������� 1

1.1 Machine Learning����������������������������������������������������������������������������������������� 5
1.1.1 Experience�������������������������������������������������������������������������������������������� 5
1.1.2 Task������������������������������������������������������������������������������������������������������ 7
1.1.3 Performance Measure�������������������������������������������������������������������������� 8
1.2 Machine Learning Paradigms���������������������������������������������������������������������� 10
1.2.1 Supervised Learning���������������������������������������������������������������������������� 10
1.2.2 Unsupervised Learning������������������������������������������������������������������������ 11
1.2.3 Semi-supervised Learning������������������������������������������������������������������ 12
1.2.4 Reinforcement Learning���������������������������������������������������������������������� 12
1.3 Maximum Likelihood Estimation������������������������������������������������������������������ 14
1.4 Elements of a Machine Learning Algorithm������������������������������������������������� 18
1.4.1 Data����������������������������������������������������������������������������������������������������� 18
1.4.2 Models������������������������������������������������������������������������������������������������� 23
1.4.3 Loss Function�������������������������������������������������������������������������������������� 25
1.4.4 Optimizer��������������������������������������������������������������������������������������������� 26
1.4.5 Regularizer������������������������������������������������������������������������������������������ 28
1.5 Bias and Variance Trade-Off������������������������������������������������������������������������� 28

iii
Table of Contents

1.6 Why Deep Learning?������������������������������������������������������������������������������������ 30

1.6.1 Curse of Dimensionality����������������������������������������������������������������������� 30
1.6.2 Invalid Smoothness Assumption���������������������������������������������������������� 32
1.6.3 Deep Learning Advantages������������������������������������������������������������������ 33
1.7 Summary����������������������������������������������������������������������������������������������������� 35

Chapter 2: Essential Math����������������������������������������������������������������� 37

2.1 Linear Algebra���������������������������������������������������������������������������������������������� 38
2.1.1 Matrices and Vectors��������������������������������������������������������������������������� 38
2.1.2 Unary Matrix Operations���������������������������������������������������������������������� 40
2.1.3 Binary Matrix Operations��������������������������������������������������������������������� 41
2.1.4 Norms�������������������������������������������������������������������������������������������������� 44
2.2 Probability Theory���������������������������������������������������������������������������������������� 45
2.2.1 Joint Probability����������������������������������������������������������������������������������� 48
2.2.2 Conditional Probability������������������������������������������������������������������������� 48
2.2.3 Elementary Rules�������������������������������������������������������������������������������� 49
2.2.4 Chain Rule������������������������������������������������������������������������������������������� 50
2.2.5 Bayes Rule������������������������������������������������������������������������������������������� 52
2.3 Differential Calculus������������������������������������������������������������������������������������� 53
2.3.1 Function����������������������������������������������������������������������������������������������� 53
2.3.2 Differentiation of Univariate Function�������������������������������������������������� 57
2.3.3 Differentiation of Multivariate Function����������������������������������������������� 60
2.3.4 Differentiation of Vector Function�������������������������������������������������������� 62
2.3.5 Differentiation of Matrix Function�������������������������������������������������������� 63
2.4 Summary����������������������������������������������������������������������������������������������������� 65

Chapter 3: Differentiable Programming��������������������������������������������� 67

3.1 Swift is Everywhere������������������������������������������������������������������������������������� 68
3.2 Swift for TensorFlow������������������������������������������������������������������������������������ 70

iv
 Table of Contents

3.3 Algorithmic Differentiation��������������������������������������������������������������������������� 73

3.3.1 Programming Approaches������������������������������������������������������������������� 73
3.3.2 Accumulation Modes��������������������������������������������������������������������������� 75
3.3.3 Implementation Approaches���������������������������������������������������������������� 82
3.4 Swift Language�������������������������������������������������������������������������������������������� 86
3.4.1 Values�������������������������������������������������������������������������������������������������� 87
3.4.2 Collections������������������������������������������������������������������������������������������� 89
3.4.3 Control Flow���������������������������������������������������������������������������������������� 92
3.4.4 Closures and Functions����������������������������������������������������������������������100
3.4.5 Custom Types�������������������������������������������������������������������������������������105
3.4.6 Modern Features��������������������������������������������������������������������������������111
3.4.7 Error Handling������������������������������������������������������������������������������������119
3.4.8 Advanced Operators���������������������������������������������������������������������������121
3.4.9 Differentiation�������������������������������������������������������������������������������������122
3.5 Python Interoperability��������������������������������������������������������������������������������135
3.6 Summary����������������������������������������������������������������������������������������������������141

Chapter 4: TensorFlow Basics����������������������������������������������������������143

4.1 Tensor���������������������������������������������������������������������������������������������������������144
4.2 Dataset Loading������������������������������������������������������������������������������������������148
4.2.1 Epochs and Batches���������������������������������������������������������������������������148
4.3 Defining Model��������������������������������������������������������������������������������������������151
4.3.1 Neural Network Protocols�������������������������������������������������������������������151
4.3.2 Sequence of Layers����������������������������������������������������������������������������153
4.4 Training and Testing������������������������������������������������������������������������������������154
4.4.1 Checkpointing�������������������������������������������������������������������������������������154
4.4.2 Model Optimization�����������������������������������������������������������������������������155
4.4.3 TrainingLoop���������������������������������������������������������������������������������������159

v
Table of Contents

4.5 From Scratch for Research�������������������������������������������������������������������������161

4.5.1 Layer���������������������������������������������������������������������������������������������������162
4.5.2 Activation Function�����������������������������������������������������������������������������163
4.5.3 Loss Function�������������������������������������������������������������������������������������165
4.5.4 Optimizer��������������������������������������������������������������������������������������������166
4.6 Summary����������������������������������������������������������������������������������������������������168

Chapter 5: Neural Networks�������������������������������������������������������������171

5.1 Gradient-Based Optimization����������������������������������������������������������������������171
5.1.1 Maxima, Minima, and Saddle Points��������������������������������������������������172
5.1.2 Input Optimization������������������������������������������������������������������������������175
5.1.3 Parameters Optimization��������������������������������������������������������������������180
5.2 Linear Models���������������������������������������������������������������������������������������������189
5.2.1 Regression������������������������������������������������������������������������������������������190
5.2.2 Classification��������������������������������������������������������������������������������������198
5.3 Deep Neural Network����������������������������������������������������������������������������������202
5.3.1 Dense Neural Network������������������������������������������������������������������������203
5.4 Activation Functions������������������������������������������������������������������������������������205
5.4.1 Sigmoid����������������������������������������������������������������������������������������������206
5.4.2 Softmax����������������������������������������������������������������������������������������������210
5.4.3 ReLU���������������������������������������������������������������������������������������������������210
5.4.4 ELU�����������������������������������������������������������������������������������������������������211
5.4.5 Leaky ReLU�����������������������������������������������������������������������������������������213
5.4.6 SELU���������������������������������������������������������������������������������������������������214
5.5 Loss Functions��������������������������������������������������������������������������������������������216
5.5.1 Sum of Squares����������������������������������������������������������������������������������216
5.5.2 Sigmoid Cross-Entropy�����������������������������������������������������������������������217
5.5.3 Softmax Cross-Entropy�����������������������������������������������������������������������218

vi
 Table of Contents

5.6 Optimization������������������������������������������������������������������������������������������������219
5.6.1 Gradient Descent��������������������������������������������������������������������������������220
5.6.2 Momentum�����������������������������������������������������������������������������������������223
5.7 Regularization���������������������������������������������������������������������������������������������223
5.7.1 Dataset�����������������������������������������������������������������������������������������������224
5.7.2 Architecture����������������������������������������������������������������������������������������225
5.7.3 Loss Function�������������������������������������������������������������������������������������227
5.7.4 Optimization���������������������������������������������������������������������������������������228
5.8 Summary����������������������������������������������������������������������������������������������������230

Chapter 6: Computer Vision��������������������������������������������������������������231

6.1 Convolutional Neural Network��������������������������������������������������������������������231
6.1.1 Convolution Layer�������������������������������������������������������������������������������234
6.1.2 Dimensions Calculation����������������������������������������������������������������������237
6.1.3 Pooling Layer��������������������������������������������������������������������������������������243
6.1.4 Upsampling�����������������������������������������������������������������������������������������245
6.2 Prominent Features�������������������������������������������������������������������������������������247
6.2.1 Local Connectivity������������������������������������������������������������������������������248
6.2.2 Parameter Sharing�����������������������������������������������������������������������������248
6.2.3 Translation Equivariance���������������������������������������������������������������������249
6.3 Shortcut Connection�����������������������������������������������������������������������������������250
6.4 Image Recognition��������������������������������������������������������������������������������������253
6.5 Conclusion��������������������������������������������������������������������������������������������������267

References����������������������������������������������������������������������������������������269
Index�������������������������������������������������������������������������������������������������283

vii
About the Author
Rahul Bhalley is an independent machine intelligence researcher. He
was the co-founder of a short-lived deep learning startup in 2018. He
has published research papers in areas such as speech processing and
generative modeling. He actively contributes to open source projects
related to deep learning on GitHub. He has also worked with Apple’s Swift
and shares Google’s vision of making it easy for others to understand deep
learning with Swift.

ix
About the Technical Reviewer
Vishwesh Ravi Shrimali graduated from BITS Pilani in 2018, where he
studied mechanical engineering. Since then, he has worked with Big
Vision LLC on deep learning and computer vision and was involved in
creating official OpenCV AI courses. Currently, he is working at Mercedes-­
Benz Research and Development India Pvt. Ltd. He has a keen interest
in programming and AI and has applied that interest in mechanical
engineering projects. He has also written multiple blogs on OpenCV and
deep learning on Learn OpenCV, a leading blog on computer vision. He
has also coauthored Machine Learning for OpenCV (second edition) by
Packt. When he is not writing blogs or working on projects, he likes to go
on long walks or play his acoustic guitar.

xi
Preface
As a programmer and student pursuing graduation, I had a lot of trouble
understanding deep learning by myself when I started this journey back
in 2015. So I decided to write a deep learning programming book that
might help people in a similar situation as mine to easily understand deep
learning. I have tried to keep the explanation of difficult deep learning
concepts simple throughout the book.
But why Swift for deep learning? Swift is a powerful general-purpose
differentiable programming language. Swift is a well-researched programming
language and the code written in it seems like reading English sentences,
making it easy for newcomers to learn programming and even deep learning.
Furthermore, Swift is optimized for performance, so researchers can write all
deep learning algorithms in a single language with simple syntax.
And what about Python’s wide ecosystem of various libraries? With the
Python interoperability feature of Swift for TensorFlow (S4TF), you can use
Python libraries within Swift!
The intended audience of this book is as follows:

1. Newcomers to programming and/or deep learning,

expert programmers, and deep learning researchers

2. People who desire to work in user space instead of

framework space with the same programming language
in deep learning without compromising the speed

For newcomers, I hope this book serves as a good starting point to

learn programming and deep learning. And for researchers, I hope this
book will make the adoption of Swift for TensorFlow for deep learning
research easy.

xiii
CHAPTER 1

Machine Learning
Basics
We’re unquestionably in the business of forging the gods.1
—Pamela McCorduck

Nowadays, artificial intelligence (AI) is one of the most fascinating fields of

computer science in addition to quantum computing (Preskill, 2018) and
blockchain (Nakamoto, 2008). Hype since the mid-2000s in the industrial
community has led to large amounts of investments for AI startups.
Globally leading technology companies such as Apple, Google, Amazon,
Facebook, and Microsoft, just to name a few, are quickly acquiring talented
AI startups from all over the world to accelerate AI research and, in turn,
improve their own products.
Consider a portable device like Apple Watch. It uses machine
intelligence to analyze your real-time motion sensory data to track your
steps, standing hours, swimming reps, sleep time, and more. It also
calculates your heart rate from temporal blood color variations underneath
your wrist’s skin, alerts you about your heartbeat irregularities, performs
electrocardiography (ECG), measures oxygen consumption in blood
(VO max) during exercise, and much more. On the other hand, devices like

1
Quoted from (McCorduck, 2004).
© Rahul Bhalley 2021 1
R. Bhalley, Deep Learning with Swift for TensorFlow,
https://doi.org/10.1007/978-1-4842-6330-3_1
Chapter 1 Machine Learning Basics

iPhone and iPad use the LIDAR information from camera sensors to create
depth map of surrounding instantly. This information is then combined
with machine intelligence to deliver computational photography features
such as bokeh effect with adjustable strength, immersive augmented
reality (AR) features such as reflection and lighting of surrounding on AR
objects, object occlusions when humans enter in the scene, and much
more. Personal voice assistant like Siri understands your speech allowing
you to do various tasks such as controlling your home accessories, playing
music on HomePod, calling and texting people, and more. The machine
intelligence technology becomes possible due to fast graphics processing
unit (GPU). Nowadays GPU on portable devices are fast enough to process
user’s data without having to send it to the cloud servers. This approach
helps in keeping the user’s data private and hence secure from undesirable
exposure and usage (Sharma and Bhalley, 2016). In fact, all the features
mentioned above are made available with on-device machine intelligence.
It might surprise you that AI is not a new technology. It actually dates
back to the 1940s, and it was not considered useful and cool at all. It had
many ups and downs. The AI technology arose to popularity for mainly
three times. It had different names over these eras, and now we popularly
know it as deep learning. Between the 1940s-1960s, AI was known as
“cybernetics”; around the 1980s–1990s, it was known as “connectionism”;
and since 2006, we know AI as “deep learning.”
At some point in the past, there was also a misconception, believed
by many researchers, that if all the rules of the way everything in the
universe works were programmed in a computing machine, then it would
automatically become intelligent. But this idea is strongly challenged by
the current state of AI because we now know there are simpler ways to
make machines mimic human-like intelligence.
In earlier days of AI research, the data was sparsely available. The
computational machines were also slow. These were one of the main
factors that drowned the popularity of AI systems. But now we have the
Internet, and a very large fraction of the population on Earth interacts with
one another which generates humongous amounts of data quickly which
2
 Chapter 1 Machine Learning Basics

are stored in servers of respective companies. (Raina et al., 2009) figured

out a way to run the deep learning algorithms with faster speed. The
combination of large datasets and high-performance computing (HPC)
has led researchers to quickly advance the state-of-the-­art deep learning
algorithms. And this book is focused on introducing you to these advanced
algorithms starting from the simpler concepts.
In this chapter, we will introduce the basic concepts of machine
learning which remain valid for its successor, the deep learning field.
Chapter 2 focuses on the mathematics required to clearly understand
the deep learning algorithms. Because deep learning is an empirical
subject, understanding only mathematical equations for deep learning
algorithms is of no use if we cannot program them ourselves. Moreover,
the computers were built to test theorems of mathematics by performing
numerical computation (Turing, 1936). Chapter 3 introduces a powerful,
compiled, and fast programming language for deep learning called Swift for
TensorFlow which extends Apple’s Swift language (which is already capable
of differentiable programming) to include deep learning–specific features
with the TensorFlow library. TensorFlow is a deep learning–specific library
and deserves the whole Chapter 4 dedicated to its introduction. Then
we dive into the basics of neural networks in Chapter 5. Finally, we will
program some advanced computer vision algorithms in Chapter 6.
But let us first differentiate between the terms artificial intelligence,
machine learning, and deep learning because these are sometimes used
interchangeably. Artificial intelligence, also called machine intelligence,
represents a set of algorithms which can be used to make machines
intelligent. AI systems usually contain hard-coded rules that a program
follows to make some sense out of the data (Russell & Norvig, 2002), for
instance, finding a noun in a sentence using hard-coded English grammar
rules, preventing a robot from falling into a pit using if and else conditions,
and so on. These systems are considered weakly intelligent nowadays.
Another term is machine learning (ML) which unlike AI algorithms uses
data to draw insights from it (Bishop, 2006), for instance, classifying
an image using non-parametric algorithms like k-nearest neighbors,
3
Chapter 1 Machine Learning Basics

classifying a text using decision tree methods, and so on. ML uses data to
learn and is also known to perform weaker than deep learning. Finally, the
current state-of-the-art AI is deep learning. Deep learning (DL) also uses
data for learning but in a hierarchical fashion (LeCun et al., 2015) taking
inspiration from the brain. DL algorithms can learn the mapping of very
complicated datasets easily without compromising accuracy, but they
instead perform better than machine learning algorithms. If you’d draw a
Venn diagram, shown in Figure 1-1, you’d see deep learning is a subset of
machine learning, whereas the artificial intelligence field is a superset of
both these fields.

Examples: prior knowledge systems,

probabilistic reasoning systems, A* search, etc.

Machine Learning
Examples: k-nearest neighbors, k-means
clustering, Gaussian processes, principle
component analysis, t-SNE, etc.

Deep Learning

Examples: convolutional networks,

generative adversarial networks,
recurrent networks, transformers,
deep reinforcement learning, etc.

Figure 1-1. A Venn diagram representing the overlap (not precisely

scaled) between artificial intelligence, machine learning, and deep
learning algorithms. Each set gives a few examples of algorithms
belonging to that field.

4
 Chapter 1 Machine Learning Basics

Now we can begin our journey with deep learning starting with simple
machine learning concepts.

1.1 Machine Learning

A machine learning algorithm learns to perform some task by learning
itself from the data while improving its performance. The widely accepted
definition (Mitchell et al., 1997) of machine learning is as follows:
“A computer program is said to learn from experience E with respect to some
class of task T and performance measure P if its performance at task T, as
measured by P, improves with experience E.” The idea is to write a computer
program that can update its state via some performance measure to perform
the desired task with good performance by experiencing the available data.
No human intervention should be required to make this program learn.
Based on this definition, there are three fundamental ideas that help
us in making machines learn, namely, experience, task, and performance
measure. Each of these ideas is discussed in this section. In Section 1.4, we
will see how these ideas are expressed mathematically such that a learning
computer program can be written. Following Section 1.4, you will realize
that this simple definition forms the basis for how machines learn and that
each paradigm of machine learning discussed in the book can be implicitly
expressed in terms of this definition.
Before we proceed further, it’s important to clarify that a machine
learning algorithm is made up of various basic components. Its learning
component is called a model which is simply a mathematical function.
Now let us move on to understand these fundamental ideas.

1.1.1 Experience
The experience is multiple observations made by a model to learn to
perform a task. These observations are samples from an available dataset.
During learning, a model is always required to observe the data.

5
Chapter 1 Machine Learning Basics

The data can be in various forms such as image, video, audio, text,
tactile, and others. Each sample, also known as example, from data can be
expressed in terms of its features. For example, features of an image sample
are its pixels where each pixel consists of red, green, and blue color values.
The different brightness value of all these colors together represents a
single color in the visible range of the spectrum (which our eyes can
perceive) of electromagnetic radiations.
In addition to the features, each sample sometimes might also contain
a corresponding label vector, also known as target vector, which represents
the class to which the sample belongs. For instance, a fish image sample
might have a corresponding label vector that represents a fish. The label
is usually expressed in one-hot encoding (also called 1-of-k coding where
k is the number of classes), a representation where only a single index
in a whole vector has a value of one and all others are set to zero. Each
index is assumed to represent a certain class, and the index whose value
is one is assumed to represent the class to which the sample belongs. For
instance, assume the [1 0 0] vector represents a dog, whereas [0 1 0] and
[0 0 1] vectors represent a fish and a bird, respectively. This means that all
the image samples of birds have a corresponding label vector [0 0 1] and
likewise dog and fish image samples will have their own labels.
The features of samples we have listed previously are raw features, that
is, these are not handpicked by humans. Sometimes, in machine learning,
feature selection plays an important role in the performance of the model.
For instance, a high-resolution image will be slower to process than its
low-resolution counterpart for a task like face recognition. Because deep
learning can work directly on raw data with great performance,
we won’t discuss feature selection in particular. But we will go through
some preprocessing techniques as the need arises in code listings to get
the data in correct format. We refer the interested readers to (Theodoridis
and Koutroumbas, 2009) textbook to read about feature selection.
In deep learning, we may require to preprocess the data. Preprocessing
is a sequence of functions applied on raw samples to transform them
into a desired specific form. This desired form is usually decided based
6
 Chapter 1 Machine Learning Basics

on the design of the model and the task at hand. For instance, a raw
audio waveform sampled at 16 KHz has 16,384 samples per second
expressed as a vector. For even a short audio recording, say 5 seconds,
this vector’s dimension size will become very large, that is, an 81,920
elements long vector! This will take longer to process by our model. This
is where preprocessing becomes helpful. We can then preprocess each
raw audio waveform sample with the fast Fourier transform (Heideman
et al., 1985) function to transform it into a spectrogram representation.
Now this image can be processed much faster than the previous lengthy
raw audio waveform. There are different ways to preprocess the data, and
the choice depends on the model design and the task at hand. We will
cover some preprocessing steps in the book for different kinds of data,
non-exhaustively, wherever the need occurs.

1.1.2 Task
The task is an act of processing the sample features by the model to return
the correct label for the sample. There are fundamentally two tasks for which
machine learning models are designed, namely, regression and classification.
There are more interesting tasks which we will introduce and program in later
chapters and are simply the extension of these two basic tasks.
For instance, for a fish image, the model should return the [0 1 0]
vector. Because here the image is being mapped to its label, this task is
commonly known as image classification. This serves as a simple example
for a classification task.
A good example of a regression task is object detection. We might want
to detect the location of an object, say ball, in an image. Here, features are
image pixels, and the labels are the coordinates of an object in the image.
These coordinates represent a bounding box for the object, that is, the
location where the object is present in a given image. Here, our goal is to

7
Chapter 1 Machine Learning Basics

train a model that takes image features as input and predicts the correct
bonding box coordinates for an object. Because the prediction output is
real-valued, object detection is considered as a regression task.

1.1.3 Performance Measure

Once we have designed a model to perform a task, the next step is to make
it learn and evaluate its performance on the given task. For evaluation, a
performance measure (or metric) of some form is used. A performance
metric can take various forms such as accuracy, F1 score, precision and
recall, and others, to describe how well the model performs a task. Note
that the same performance metric should be used to evaluate the model
during both training and testing phases.
As a rule of thumb, one must try to select a single-number
performance metric whenever possible. In our previous image
classification example, one can easily use accuracy as a performance
metric. Accuracy is defined as a fraction of the total number of images
(or other samples) classified correctly by the model. As shown next, a
multiple-number performance metric can also be used, but it makes it
harder to decide which model performs best from a set of trained models.
Let us consider two image classifiers C1 and C2 whose task is to predict
if an image contains a car or not. As shown in Table 1-1, if classifier C1 has
0.92 accuracy and classifier C2 has 0.99 accuracy, then it is obvious that C2
performs better than C1.

Table 1-1. The accuracies of classifiers C1 and

C2 on an image recognition task.

Classifier Accuracy

C1 92%
C2 99%

8
 Chapter 1 Machine Learning Basics

Now let us consider precision and recall for these two classifiers which
is a two-number evaluation metric. Precision and recall are defined as a
fraction of all and car images in the test or validation set that the classifier
correctly labeled as cars, respectively. For our arbitrary classifiers, these
metric values are shown in Table 1-2.

Table 1-2. The precision and recall

of classifiers C1 and C2 on an image
recognition task.

Classifier Precision Recall

C1 98% 95%
C2 95% 90%

Now it seems unclear which model has a superior performance. We

can instead turn precision and recall into a single-number metric. There
are multiple ways to achieve this such as mean or F1 score. Here, we will
find its F1 score. F1 score, also called F-measure and F-score, is actually a
harmonic mean between precision and recall and is calculated with the
following formula:

2
F1 =
1 1 (1.1)
+
Precision Recall

Table 1-3 shows the F1 score for each classifier by putting their
precision and recall values in Equation 1.1.
From Table 1-3, by simply looking at the F1 scores, we can easily
conclude that classifier C2 performs better than C1. In practice, having
a single-number metric for evaluation can be extremely helpful in
determining the superiority of the trained models and accelerate your
research or deployment process.

9
Chapter 1 Machine Learning Basics

Table 1-3. The precision, recall, and F1 score of classifiers C1 and C2

on an image recognition task.
Classifier Precision Recall F1 score

C1 98% 85% 91%

C2 95% 90% 92.4%

Having discussed the fundamental ideas of machine learning, we will

now shift our focus toward different machine learning paradigms.

1.2 Machine Learning Paradigms

Machine learning is usually classified into four categories based on the
kind of dataset experience a model is allowed as follows: supervised
learning (SL), unsupervised learning (UL), semi-supervised learning (SSL),
and reinforcement learning (RL). We briefly discuss each of these machine
learning paradigms.

1.2.1 Supervised Learning

During training, when a model makes use of labeled data samples for
learning to perform a task, then this type of machine learning is known
as supervised learning. It is called “supervised” because each sample
belonging to the dataset has a corresponding label. In supervised learning,
during training, the goal of the machine learning model is to map from
samples to their corresponding targets. During inference, the supervised
model must predict the correct labels for any given samples including
samples unseen during training.

10
 Chapter 1 Machine Learning Basics

We have already gone through an idea of an image classification task

previously which is an example of SL. For example, you can search photos
by typing the class (or category) of object present in the photo in Apple’s
Photos app. Another interesting SL task is automatic speech recognition
(ASR) where a sequence of audio waveforms is transcribed by the model
into a textual sequence representing the words spoken in the audio
recording. For instance, Siri, Google Assistant, Cortana, and other personal
voice assistants on portable devices all use speech recognition to convert
your spoken words into text. At the time of writing, SL is the most successful
and widely used machine learning in production.

1.2.2 Unsupervised Learning

Unsupervised learning is a type of machine learning where a model is
allowed to observe only sample features and not the labels. UL usually
aims at learning some useful representation of a dataset in the hidden
features of model. This learned representation can later be used to perform
any desired task with this model. UL is of great interest to the deep learning
community at the time of this writing.
As an example, UL can be used to reduce the dimensionality of high-­
dimensional data samples which, as we discussed earlier, can help in
processing the data samples faster through the model. Another example is
density estimation where the goal is to estimate the probability density of
a dataset. After density estimation, the model can produce samples similar
to those belonging to the dataset it was trained on. UL algorithms can be
used to perform various interesting tasks as we shall see later.
It’s very important to note that UL is called “unsupervised” because of
the fact that labels aren’t present in the dataset but we still require labels to be
fed to the loss function (which is the fundamental requirement of maximum
likelihood estimation discussed in Section 1.3) along with the prediction in
order to train the model. In this situation we assume some appropriate labels
for the samples ourselves. For example, in generative adversarial networks

11
Chapter 1 Machine Learning Basics

(Goodfellow et al., 2014), the label for datapoint generated from a generator is
given a fake label (or 0), whereas a datapoint sampled from a dataset is given
a real label (or 1). Another example is auto-encoder (Vincent et al., 2008)
where labels are the corresponding sample images themselves.

1.2.3 Semi-supervised Learning

Semi-supervised learning is concerned with training a model from a small
set of labeled samples and the predictions (using the contemporarily semi-
trained model) for unlabeled samples as pseudo-targets (also called soft
targets) during training. From the perspective of the kind of data experienced
during training, SSL is halfway between supervised and unsupervised
learning because it observes both labeled and unlabeled samples. SSL is
particularly useful when we have a large dataset containing only a handful of
labeled samples (because they’re laborious and hence costly to obtain) and
a large number of unlabeled samples. Interestingly, an SSL technique for
training the model can considerably boost its performance.
We do not cover semi-supervised learning in the book. For rigorous
understanding of semi-supervised learning, we refer the interested readers
to (Chapelle et al., 2006) textbook.

1.2.4 Reinforcement Learning

Reinforcement learning is based on reward obtained by the agent
interacting with the environment to maximize its cumulative reward
(weighted average sequence of rewards, also known as return) over a
number of trials (called episodes) to achieve its goal. RL is a paradigm of
machine learning that involves a sequence of decision-making processes.
The agent acts on the world by taking some action, say it moves
forward. Following this, the environment’s state gets updated, and a
reward is returned (or given) back to the agent by the environment.

12
 Chapter 1 Machine Learning Basics

The reward is either positive or negative and can be, respectively, regarded
as a good or bad response to the agent from the world in accordance to the
behavioral science viewpoint. We are more interested in return instead of
the current step reward because the goal of the agent is to maximize the
return over the course of each episode. Here, an episode is a sequence of
interactions between an agent and its environment from a start to an end.
Examples of an episode are as follows: a gameplay by the agent where the
game ends when a certain condition is met and an agent trying to stay
alive in harsh environmental conditions until it dies due to some accident.
See Figure 1-2 for a diagrammatic view of an interaction between the agent
and its environment.

State St

Reward Rt Agent

Action At

Environment

Figure 1-2. An interaction between a reinforcement learning agent

and the environment.

13
Chapter 1 Machine Learning Basics

The agent perceives the previous state of the environment St-1 and
takes an action At on the environment whose state St changes and is
returned to the agent. The agent also receives a scalar reward Rt from
the environment describing how good is the current state for the agent.
Although the state of the environment changes when the agent acts, it may
also change by itself. In multi-agent reinforcement learning, there might
also be other agents maximizing their own returns.
Reinforcement learning is a very interesting machine learning field
and is being studied aggressively at the time of this writing. It is also
considered to be more close to the way humans (or other mammals) learn
by making behavioral modifications, that is, reinforcing an action based on
the reward. A recent work (Minh et al., 2015) showed that a combination
of deep learning and reinforcement learning called deep reinforcement
learning can even surpass human-level gameplay capabilities.
Unfortunately, we don’t discuss reinforcement learning in the book.
Interested readers may refer (Sutton and Barto, 2018) textbook for the
fundamentals of this field. For deep reinforcement learning advances, we
suggest the works (Mnih et al., 2015; Schulman et al., 2017).
Now let us look at the basic idea known as maximum likelihood
estimation that helps in constructing machine learning algorithms.

1.3 Maximum Likelihood Estimation

The setting described here is assumed throughout the machine learning
literature. Constrained with this setting, the parameters of a model are
estimated. There are two fundamental approaches to solve the parameters
estimation problem, namely, maximum likelihood estimation and Bayesian
inference. We will focus on maximum likelihood estimation because
this is what we’ll use to train neural networks throughout the book. The
readers interested in Bayesian inference are suggested to read Chapter 2
of (Bishop, 1995) textbook. And for detailed notes on the origination of the
maximum likelihood function, please refer (Akaike, 1973).

14
 Chapter 1 Machine Learning Basics

To solve a machine learning problem, we require a dataset ⅅ

{(
containing a set of N datapoints (or samples), that is, ⅅ = x (i ) ,t (i )
Assume that each datapoint is identical and sampled independently from
)}
i =1
N
.

a joint data-generating distribution Pd(x, t) which is a probability density

function (PDF) meaning that data sample x and corresponding target
t are continuous random variables. Note that the distribution of target
random variable t is actually dependent on the task performed by the
model, that is, the target’s distribution is, respectively, continuous and
discrete for regression and classification tasks. We will also call Pd(x, t) as
a data probability density function (or data PDF). We only have access to
the dataset ⅅ sampled from the data PDF and not the distribution itself.
Therefore, we cannot access more datapoints than what are available to us.
Because the dataset ⅅ is sampled from the data PDF, it statistically
describes the data PDF itself. Our goal in parametric machine learning is
to approximate the mapping of the data PDF by estimating the parameters
of a parameterized probability density function Pm(x, y|θ) (or simply
Pm(x, y)) of our own choice which we will call a model probability density
function (or model PDF), and we will also frequently refer to it simply
as a model throughout the book. Here, θ represents parameters, and the
random variables x and y are the data sample and the corresponding
predicted value given x as input. We will actually minimize the distance,
also called loss or error, between prediction variable y and target variable t
by updating the parameter values using the maximum likelihood function.
The model and data PDFs are completely different distributions
such that their samples are also statistically different. But we want the
predictions y from the model PDF to resemble corresponding targets t
from the data PDF for a given data sample x. As pointed out earlier, we
don’t have access to the parameters of the data PDF so we cannot simply
copy its parameter values to those of the model function. But we do have
dataset 𝒳 at our disposal which we can exploit to approximate the data
PDF because it’s its statistical description.

15
Chapter 1 Machine Learning Basics

Now we will describe maximum likelihood estimation to approximate

the data PDF with our parameterized model PDF. Because the datapoints
in 𝒳 are independent and identically distributed, their joint probability is
given by

( )
N
Pm ( ⅅ q ) = Õ Pm x (i ) , t (i ) q =  (q ⅅ)
i =1 (1.2)

Here, Pm(ⅅ|θ) is the conditional model PDF and is read as “joint

probability of a dataset given the parameters.” The function ℒ(θ|ⅅ) is a
likelihood function of parameters θ for a given fixed and finite dataset ⅅ.
You may also interpret this function as “likely a good approximation of the
data PDF from which the dataset ⅅ is sampled.” To reduce the clutter, we
will implicitly assume the parameters conditioning in both the model and
log-likelihood function. We will use the Bayes theorem to convert joint
data probability distribution into a conditional distribution we care about.

( ) ( ) ( )
N N
 = Õ Pm x (i ) , t (i ) = Õ Pm t (i ) x (i ) Pm x (i ) (1.3)
i =1 i =1

The data PDF is approximated by maximizing the likelihood function

which requires updating the parameter values of the model PDF. More
specifically, the parameters are updated by an iterative method as briefly
described in Section 1.4 and at great length in Section 5.1. Numerically,
it is more convenient to first take the negative logarithm of the likelihood
and then minimize it. This is equivalent to maximizing the likelihood
function.

( ) ( )
N N
L = - ln  = -å ln Pm t (i ) x (i ) - å ln Pm x (i ) (1.4)
i =1 i =1

The logarithm function plays a very important role here. It turns

the products into summations which helps in stabilizing the numerical
computation. This is because the products of values closer to zero are

16
 Chapter 1 Machine Learning Basics

much smaller values which may get rounded off due to limited-precision
representational capacity of computational devices. This is why machine
learning frequently uses the logarithm function.
The negative log-likelihood function can be regarded as a loss or
error function denoted by L(.). Our goal is to minimize the loss function
by updating its parameter values such that our parameterized model
PDF approximates the data PDF using the available fixed and finite data
samples from the data PDF. Note that the second term in this equation
doesn’t contribute in the parameters estimation of the model PDF
because it doesn’t depend on the model’s parameters and is simply a
negative additive term which can be ignored for maximizing the likelihood
function. The loss function can now be simply described by the following
equation:

( )
N
L = -å ln Pm t (i ) x (i ) (1.5)
i =1

Minimizing the loss function is equivalent to minimizing the negative

log-likelihood function which further can be considered as maximizing the
log-likelihood function, hence the term maximum likelihood estimation.
Here, our model PDF represents a conditional distribution of targets given
the data samples Pm(t|x). We shall see later that a neural network is a
framework for modeling this conditional distribution. And also based on
the distribution of the target random variable, different loss functions arise
using this equation.
As a sidenote, hypothetically speaking, if the likelihood function
perfectly approximates the data PDF, then the available dataset 𝒳 and
other identical datasets can be sampled from it. But in practice, it is not
possible to perfectly approximate the data PDF, but we can instead closely
approximate it. This is all what we do in machine learning.

17
Chapter 1 Machine Learning Basics

Now let us look at the elements of a machine learning algorithm and

make the vague definition of machine learning given by (Mitchell et al.,
1997) mathematically concrete.

1.4 E lements of a Machine Learning

Algorithm
In this section, we describe the fundamental elements of a machine
learning algorithm that apply to all paradigms of machine learning briefly
discussed in Section 1.2. There are four crucial components of a machine
learning algorithm, namely, data, model, loss function, and optimization.
Optionally, one can also use regularization techniques to improve the
generalization of the model and balance the bias and variance trade-off
(see Section 1.5).
Remember that in machine learning, our primary goal is to train a
model that should perform well on unseen data because our training
dataset will not contain the data generated by users in the future.

1.4.1 Data
The data present in the dataset serves as an experience for the machine
learning model. When the model is first initialized with random parameter
values, it has no knowledge of how to perform well on the certain task.
This knowledge is gained by iteratively exposing the model to data (usually
in small sample counts, also known as mini-batches). As the model
experiences more samples, it gradually learns to perform the task more
accurately.
A dataset is simply a structured, usually a tabular (or matrix),
arrangement of datapoints. Table 1-4 shows an arbitrary example of
a dataset. This dataset contains some characteristics (or features) of
people where each row contains the features for a single person. Each

18
 Chapter 1 Machine Learning Basics

row contains the height (in centimeters), age (in years), and weight
(in kilograms) values for a certain person. Note that features and
their corresponding targets can be tensor values of any dimensions (0
dimensions, i.e., scalar variables, here).

Table 1-4. Dataset of people with their

heights, ages, and weights represented by
x1, x2, and t scalars, respectively.
Height (x1) Age (x2) Weight (t )

151.7 25 47.8
139.7 20 36.4
136.5 18 31.8
156.8 28 53.0

Given the dataset, we must decide the features and targets for a task.
In other words, selection of the correct features for a task is dependent
solely upon our decision. For instance, we can use this dataset to perform
a regression task of predicting the weight given the height and age of a
person. In this setting, our feature vector x contains height x1 and age x2,
that is, x = [x1 x2], for each person (called sample), whereas the target t
is the weight of a person. For example, x = [136.5 18] and t = 31.8 are the
feature vector and target scalar for the third person in the given dataset.
In machine learning literature, datapoint input to the model is also
known as example or sample, and its features are also known as attributes.
For instance, features of a car can be color, tire size, top speed, and so on.
Another example can be a drug whose features may include chemical
formula, proportion of each chemical element, and so on. The target, also
known as label or hard target (when differentiating from soft target), is the
desired output corresponding to a given sample. For instance, given the

19
Exploring the Variety of Random
Documents with Different Content
 WAYSIDE SHRINE WITH AN OFFERING OF FLOWERS

In remote caves of mountain regions it is claimed, and, I believe,

truthfully, that the ancient deities are still worshipped. It is no infrequent
occurrence to see a bouquet of flowers before the image of the virgin in
the churches or wayside shrines. Sometimes even offerings of wheat or
fruits are found, the gift of some poor peon in whose mind the
conception of the Saviour and his mission on earth is very vague.
Several writers assert that they have personally seen Indians on their
way to the mountains to sacrifice lambs, chickens and flowers to their
gods, thus proving that the grosser forms of paganism have not been
stamped out entirely. The priests, of course, do not approve of this, and
try in every way to stop these practices, but without success.
The Catholic Church used to be all-powerful in Mexico. It held the
wealth and the learning, and the priests preyed upon the people as well
as prayed for them. They were taxed to the utmost, and “Pay or pray”
was the motto affixed to the cross by the priests. Rich men gave freely
of their substance. Poor peons—and they are vastly in the majority—
went clothed in rags that the Church might be benefited. The favourite
method was by the sale of indulgences. General Thompson, United
States Minister to Mexico in 1845, wrote as follows: “As a means of
raising money, I would not give the single institution of the Catholic
religion (in Mexico) of masses and indulgences for the benefit of the
souls of the dead for the power of taxation possessed by any
government. I remember that my washerwoman once asked me to lend
her two dollars. I asked her what she wanted with it. She told me that
there was a particular mass to be said on that day which relieved the
souls in purgatory from ten thousand years’ torture and that she wished
to secure the benefit for her mother.” It is like the harangue that so
aroused Martin Luther: “The very moment the money clicks on the
bottom of this chest the soul escapes from purgatory and flies to
Heaven! Bring your money, bring money, bring money!”
Shrines and chapels were so numerous that the true believer passed
through the streets with head uncovered and hat in hand, for fear that
he might pass one unobserved and not remove his head covering as
piety demanded. During the latter years of Spanish rule in Mexico, the
Church became so enormously rich that it was reported to have in its
possession one-third of all the wealth in Mexico. In addition to the
power the Church naturally held, this immense wealth gave its leaders
great prestige in governmental affairs, for wealth everywhere commands
power and respect among those in authority. At one time the clergy held
property to the value of about $180,000,000, yielding an annual income
of $12,000,000, according to reliable authorities. Some have estimated
the wealth at more than $600,000,000.
It had secured control not only of the wealth, but also much of the
best agricultural land within the republic, owning eight hundred
haciendas and more than twenty-two thousand city lots. All this was tied
up and became useless and non-productive. The Church used its great
influence to oppose all progress. The opposition finally broke forth, and
the immense wealth of convents, shrines and monasteries was poured
forth with lavish hand in what the Church considered a holy war against
heretical ideas and persons. Reformers set envious eyes upon this
property, and numerous attempts were made to dispossess the Church
of it. An edict aimed at the power of the Church was issued by
Commonfort in 1857, but the Indian reformer and president, Juarez, was
the first to actually accomplish the separation of church and state
several years later. The establishment of the empire with Maximilian as
Emperor was simply a reaction, and an attempt to establish a
government in which the interests of the Church would again be
paramount. It is not much wonder that the native population yielded so
readily to the overthrow of the priestly power. In accomplishing the
complete overthrow of church and state, Mexico only did what Italy did
a few years later, and what France is endeavouring to do at the present
time. Even in Spain, the handwriting on the walls seems to point to the
same ultimate result. And yet it is strange to see a nation so rigidly and
even unmercifully regulating a church to which ninety-five per cent. of
the people belong.
The reactionary movement on the part of the Church under the guise
of French intervention failed. The reform anti-clerical movement
prevailed once more, even though opposed by the enormous wealth of
the Church. The greater portion of the property once owned by the
Church has been lost. The country abounds in ruined churches and
convents. The law went so far as to prohibit the Church from holding the
title to property, and if it wished to own property, it must be in the
names of individuals. Priests were forbidden, under penalty of fine and
imprisonment, to appear in the streets in their clerical dress. Religious
processions outside the walls of the church, or churchyard, were
prohibited. Civil ceremonies were made obligatory to render a marriage
valid. Sisters of Charity and the Jesuits were sent out of the country, and
even the ringing of bells was regulated by law. It has now lost not only
its property but its prestige as well.
The property was confiscated, or “denounced,” and sold for beggarly
sums in numerous instances. Many hotels are now located in former
churches or convents, and schools and barracks innumerable occupy
former homes of nuns. Even the famous prison of Belem in the City of
Mexico, where more than three thousand offenders (most of them justly
no doubt) have been incarcerated at times, was the old convent of that
name; and the military prison, Santiago de Tlalteloco, was one of the
oldest churches in Mexico, having been founded by the first viceroy.
Protestant services are held in a number of places that were former
Catholic churches, the buildings having been purchased by these
organizations, or the use of them granted by the authorities. The rich
silver plate and the altar rails were looted from the sacred edifices, or
were sold for small sums by the officers.
For many years Mexico has thus gone along the line of reform. The
ambition of the Church has been held in check but not killed. They are
regaining some of their former power, and recovering much of their
former property, so it is claimed by good authority.[4] The average
Mexican is superstitious. He is boastful and bold in times of peace, but
craven when the time of trial comes. Consequently, when sick and about
to die, he will send for the priest, no matter how he may have fought
the Church when in health. The priests, or some of them at least,
claiming that the Catholic Church, as the chosen of the Lord, has a lien
on all earthly goods, refuse to administer the sacrament without some
restitution. If the dying man owns a confiscated church property, he
must restore its value before he can get a clear title to a home in
Heaven. With the persistence characteristic of the Mexican Catholic
priests, they are ferreting out their former property and again
accumulating wealth for their beloved Church. Their fees are utterly out
of proportion to the earning capacity of the people. Marriage costs
$14.00, baptism $2.25 and plain mass $6.00. Many of the poor peons
are obliged to forego the services of the Church because of these high
charges, for all services must be paid in advance.
They are also openly disregarding the established laws in some of the
restrictions imposed. I travelled for two days on the railroad with the
Bishop of Tehuantepec who wore his purple robes of office all the time.
At nearly every station priests met him, and he was given a continuous
ovation. A few months ago, according to a Mexican periodical, a well
known priest, in defiance of the law which prohibits public religious
processions, authorized such a procession, and blessed at the altar
those who arrived with it. In many of the more remote districts the law
requiring marriage ceremonies to be made by civil authorities is
completely disregarded. The priests tell the people that the religious
ceremony is all that is necessary. Although the Church upholds such
marriages, in law they are absolutely null and void, and it is a deceit
upon the contracting parties. Some priests go so far as to tell their
people that the civil marriage is positively impious. And yet nothing is
done to punish the above violations of the established laws. The
government probably does not consider that these infractions of the
strict letter of the law have reached a serious phase.
If the Roman Church of Mexico to-day, with its wealth confiscated, its
public voice muzzled, its political powers annulled, has still power so
that it can openly violate some of the fundamental laws of the country,
we can have some faint idea of its power when it ruled the country with
an iron hand. Those who see trouble ahead because of the
avariciousness of some of the clergy, are fond of quoting the old Spanish
proverb “The devil lurks behind the cross.” Nevertheless, I believe that
the clergy in Mexico to-day are superior to those who served prior to the
change in status. Many of them are noble men striving to uplift the
people and aid the government in its campaign for the enlightenment of
the masses. The strife has purified them and they think less of the
perquisites than the duties of their office. The well meaning priest no
doubt suffers for the sins of his predecessors as well as those of his
contemporaries who are blinded by the past glory of the Church. The
Church as an institution is probably to some extent the victim of the
ignorance and fanatical zeal of its early founders in Mexico. The Church
will thrive far more when placed on the same footing as all churches are
in the United States, and people and priest accept that condition. As one
prominent American priest has recently said in commenting on the
struggle in France: “Everywhere that church and state are united, the
church is in bondage. Nowhere is the church so free and untrammelled,
or so progressive, as in the United States.”
The churches in all the cities are numerous and their capacity far
greater than the number of those attending. Puebla, the City of the
Angels, so called because of the many miraculous visits of the angels
who even, on their first visit, measured off the city and fixed the site of
some churches, is called the city of churches as it has the greatest
number in proportion to the population of any city in the republic, many
of them being erected in honour of the various angelic visitations. The
City of Mexico contains the largest and most pretentious church building
in the new world—the cathedral. It is also one of the largest church
edifices in the world. This grand cathedral begun in 1573 was ready for
service about three-quarters of a century later but the towers were not
completed until 1791. It is four hundred and twenty-six feet long and
almost two hundred feet wide with walls of great thickness, and reaches
a height of one hundred and seventy-five feet in the dome. The towers
are a little more than two hundred feet high. Then adjoining this
building is another church, the Sagrario Metropolitano, which, to all
appearances, is a part of the main structure, although of an entirely
different and less beautiful style of architecture.
Within these two edifices were concentrated for centuries the pomp
and ceremony of the Church of Rome and within their walls much of
Mexico’s history was made. It is still the headquarters of the church
party while across the plaza is the National Palace, the official home of
the government which conquered in the long struggle between the two
forces. The estimated cost of the cathedral is $2,000,000, but that
represents only a fraction of the actual cost if the labour is figured at a
fair rate and the material had all been purchased at market value. There
are some paintings by famous artists on the walls and dome. A
balustrade surrounds the choir which is made of composite metal of
gold, silver and copper and is so valuable that an offer of a speculative
American to replace it with one of equal weight in solid silver was
refused. Within the walls there are fourteen chapels dedicated to the
various saints, and candles are kept constantly burning before the
images, and in these chapels are kept many gruesome relics of these
same saints. The remains of many of the former viceroys and some of
the other noted men in Mexican history lie buried here. This, the
greatest church in the western world, is also built on the foundations of
the greatest pagan temple of the continent—the imposing Teocalli of the
Aztecs. From the top of the towers we can look upon the same valley
that Cortez viewed when Montezuma took him by the hand after
ascending the great altar, and pointed out the various places of interest.
The lakes have receded, the architecture is different, but our admiring
eyes see the same majestic hills on every side.
Listening to the bells in the towers of this cathedral, once so powerful,
one, who is a “dreamer of dreams,” can almost imagine them lamenting
the changed times in the words of the last poem written by the poet
Longfellow:
 “Is then the old faith dead,”
They say, “and in its stead
Is some new faith proclaimed,
That we are forced to remain
Naked to sun and rain,
Unsheltered and ashamed?

“Oh bring us back once more

The vanished days of yore,
When the world with faith was filled;
Bring back the fervid zeal,
The hearts of fire and steel,
The hands that believe and build.

“Then from our tower again

We will send over land and main
Our voices of command,
Like exiled kings who return
To their thrones, and the people learn
That the Priest is lord of the land!”

The very first movement on the part of Protestant organizations to

evangelize Mexico was made by the American Bible Society when they
sent out one of their representative with the American army in 1846.
This man distributed several thousand copies of the scriptures between
Vera Cruz and the capital which afterwards bore fruit. A few years later
a woman, Miss Matilda Rankin, who had been engaged in missionary
work in Texas, crossed over the border and held services in Monterey. In
1862 a Baptist missionary, Rev. James Hickey, also began work in
Monterey. However, no organized effort was made by Protestant bodies
until the years from 1869 to 1880, when missionaries were sent by the
following denominations: Protestant Episcopal, Methodist Episcopal,
Methodist Episcopal South, Presbyterian, Baptist, Christian and
Congregational. Bishop H. C. Riley obtained an old church for the use of
the Protestant Episcopal Church and Rev. William Butler purchased a
part of the convent of San Francisco, in the heart of the city, for the
Methodist Episcopal Church.
Dios y libertad had been the watchword of the reform movement, but
it had not been put into practice until the time of President Juarez, who
encouraged mission work, and exerted himself to protect the
missionaries from fanatics. However frequent attacks upon these
workers were made in provincial towns and one foreign missionary, Rev.
J. L. Stephens, of the Congregational Church, was slain at Ahualuco in
1874. A number of native converts and preachers have met with serious,
and even fatal injuries, but no other Americans have been killed.
President Diaz has also encouraged these ministers when they were
downhearted. Rev. William Butler quotes an interview which several
missionaries had with him in which the President expressed himself as
follows: “I have seen this land as none of you ever saw it, in
degradation, with everything in the line of toleration and freedom to
learn. I have watched its rise and progress to a better condition. We are
not yet all we ought to be and hope to be; but we are not what we once
were; we have risen as a people, and are now rising faster than ever. My
advice is, do not be discouraged. Keep on with your work, avoiding
topics of irritation and preaching your gospel in its own spirit.” The
president has no warmer supporters than the Protestant missionaries
and their little bands of adherents.
Their numbers to-day after thirty years of aggressive work seem
small, as the ten Protestant denominations who maintain missions in
Mexico only claim about twenty-five thousand members, or about one
hundred thousand adherents including those who attend the Sunday-
school and other services. The Presbyterians are working in fourteen
different states. They have fifty organized churches and two hundred
and twenty-two outstations which are served by twenty-one foreign
missionaries and one hundred and one native workers. The Methodist
Episcopal Church has twenty-nine missionaries in the field and one
hundred and twenty-two native workers, and is holding services at more
than a hundred different places. The various denominations have
divided up the field and are working together in harmony. The
Methodists, for instance, are working in Guanajuato, Leon, Pachuca,
Puebla, Silao, and Oaxaca. The Presbyterians have centred their efforts
in Aguas Calientes, Zacatecas, Saltillo, San Luis Potosi and Jalapa. All
denominations have missions in the City of Mexico. The Methodists,
Baptists, Presbyterians and Congregationalists have their own publishing
houses and issue periodicals and a great deal of printed matter in
Spanish. There are in all about two hundred and fifty foreign
missionaries in Mexico serving about seven hundred congregations.
Many of these workers are medical missionaries who are doing a vast
amount of good, and others are teachers who are instructing the youth.
The Protestant bodies own property in Mexico valued at nearly two
million dollars.
An era of at least tolerance toward Protestants is dawning in this land,
and religious liberty is an actual fact. The Young Men’s Christian
Association has a strong organization in the capital. A fund has recently
been raised to erect a splendid new building for the association. The
President and his cabinet have also attended some special memorial
services in the Protestant churches. This may seem a small thing, but a
quarter of a century ago it would have been incredible. Some of the
broad-minded Catholic clergy are even displaying a kindlier feeling
toward the Protestant workers. It may not be many years before
Catholic clergy and Protestant ministers may unite together in working
for a common cause—the betterment of the morals and conditions of
the people.
 CHAPTER XVIII
PASSING OF THE LAWLESS

A rude wooden cross set up in a pile of stones is one of the striking

features of Mexican landscape that is frequently seen. As the train whirls
along through a narrow pass, high up on the mountain sides the cross is
seen outlined against the sky; or, if you are pursuing your journey by
horse or mule, in the remote districts away from the railways, your
reverie is suddenly interrupted by coming upon one of these silent
sentinels unawares. These crosses are mute reminders of an age that is
passing away. Each one marks the spot where a murder has taken place
in times past. It is an appeal for the good Catholic to mutter a prayer for
the soul of the murdered one, who was thus without preparation thrust
into the world beyond. There was a time, and that not more than a
generation ago, when the murderous and lawless classes were
numerous in Mexico. The Mexican bandit was so much feared, that,
even to this day, some hesitate to travel in that country, and many more
make walking arsenals of themselves before turning their faces toward
our southern neighbour.
If the traditionary history that has come down to us is to be believed,
these robber clans can trace their lineage back to the peregrinating
merchants of the Aztecs, and Toltecs. The rich merchant of those days
travelled over the country visiting the various cities with his wares. For
self protection they were obliged to carry with them a large force of
armed retainers. This knowledge of their own power led them to
violence. If, for any reason, these merchants became angered at a town,
or, if the people refused to trade with them, they would attack it, pillage
it and carry off the inhabitants to be sold as slaves in other remote
places, or hold them for ransom. This course generally proved far more
remunerative than the more prosaic occupation of barter and trade. It
was indeed a strong town in those days that could afford to refuse to
trade with some of the powerful merchants. If one trader was not strong
enough himself, he could easily enlist the assistance of another of his
class, as the loot and slaves would be sufficient to remunerate both very
well for the undertaking.
Later came the freebooters, who, in early Spanish days, had things
very much their own way. Although many of these were well known,
they would visit the cities armed to the teeth and no one would dare to
molest them. It is even claimed, and with good reason, that many
officials were in league with these knights of the road, and gave them
information, and assisted them in their plans to waylay wealthy
inhabitants. So long as the outlaws did not interfere with matters of
government, their immunity was practically secure. There is one city in
the northern part of Mexico named Catorce, the Spanish numeral for
fourteen, because, for a long time, it was the stronghold of fourteen of
the boldest, bravest and worst bandits that Mexico ever produced, who
terrorized the country round about and could not be captured or
subdued.
After independence, came a series of revolutions and uprisings for
more than a half century. The bandits then became guerillas, fighting on
whichever side offered the greatest advantage. They would loot a
church, or rob the hacienda of some wealthy landowner, with equal
cheerfulness. The place or person robbed depended upon whether the
guerillas were enlisted in the cause of the clericals, or anti-clericals. By
reason of the many turmoils and fights that took place, these guerillas
became a numerous and powerful class with their rendezvous in the
mountains, which, in no part of Mexico, are far distant. Before the
advent of the railroads and telegraph it was a difficult matter to cope
with these robber bands in Mexico because roads were lacking, and
their haunts were almost inaccessible. This was one of the first problems
attacked by President Diaz when he came into power, and he did it with
the boldness, originality and dash for which he was noted.
This new leader found the army a disorganized band of guerillas led
by a few men, not always over-scrupulous, and many parts of the
country overrun by bands of outlaws with whom the local authorities
were utterly unable to cope. Having some veteran troops after his many
campaigns, Diaz sent them after the bandits whenever opportunity
afforded. They were hunted and trailed into their mountain fastnesses.
The soldiers were instructed never to take captives. A little heap of fresh
dirt, or a few stones, marked the place where a living and breathing
bandit had once stood. This war of extermination made welcome to
many the proposition of Diaz. This was that he would furnish
employment to those outlaws who should surrender, and would grant to
them protection. The President being known as a man of his word, this
proclamation had its effect and large numbers formerly under the ban of
law, surrendered.

A RURALE

From this class of men the first companies of rurales were formed.
Finding it was more profitable, or at least safer, to be in favour with this
aggressive government than under its ban, they willingly entered this
service. These men were brave and thoroughly familiar with all the
mountain retreats and haunts of the outlaw bands. They hunted down
their former confederates until a live bandit was a rare specimen.
Travelling once more became secure, and now there are few places in
Mexico where it is not perfectly safe for a traveller to journey. The
companies of rurales, of which there are many, form one of the most
effective forces for preserving order ever devised by any government.
Like the famous guardia civil of Spain, the rurales patrol the remote
mountain trails and great plains of the central plateaus, and are in
reality a body of rural police. Many a lonely traveller has been made glad
by the sight of the gray uniform of this band. They are generally kind
hearted, and will do everything in their power for a foreigner. Their
uniform is the typical riding costume of the country, and differs from the
French appearance of the uniforms of the regular army. They are fine
horsemen, expert in the use of pistol and carbine, and form one of the
most picturesque cavalry bodies in the world.
There is no sickly sentimentality wasted upon law breakers, and the
highwayman, or robber, gets little sympathy. Few criminals get a second
opportunity to commit their outrages through the pardoning process.
The old ley fuga, or law of attempted escape, which was in force under
Spanish rule, under which Indians or slaves attempting to flee were
shot, was revived. Orders were promulgated to shoot highwaymen on
sight, and all other prisoners if escape was attempted. Few attempts to
escape are now made by prisoners, for the guards have a reckless way
of sending bullets after fleeing prisoners, so that no chains are needed
to secure them. The bullets are swift and any one in custody, even
though held as a witness, will be followed by the quick, death-dealing
messengers, if an attempt to escape is made. Gangs of convicts may be
seen in various places working on the streets, or on the roads, under
military guard but without shackle. The only report necessary in the
event a prisoner is killed is that he attempted to escape. It may be a
harsh proceeding, but it saves the state a great deal of money, and
conviction is sure. Furthermore, it relieves judge, jury and prison officials
of much hard work and annoyance.
A few years ago the Mexican army consisted of a few thousand
irregular, nondescript soldiers so common in Spanish-American
countries. Such men it was who placed Porfirio Diaz in power in 1876,
the same year that we were celebrating the first centennial of our
independence. In promoting peace this man of Mexico has not forgotten
the arts of war. The army has been improved until it has ceased to be
made up of the comic-opera type of the barefooted, half-naked soldier,
and is now a well fed, well equipped, and well clothed organization to
which Mexicans can point with pride. To the American eye the soldiers
appear rather indifferent and insignificant, because of their smaller
stature and brown skin, which reveals the fact that the regular soldier is
generally drawn from the lower classes of Mexicans.
Although Mexico might be termed a military nation, as military service
is made obligatory by the law of the country, yet in times of peace this
service is not enforced. It is said that the majority of the enlistments are
not even voluntary, but that recruits are drawn from the ranks of those
who are persistent law breakers—those guilty of petty criminal offences
which we would term misdemeanours. Many of these peon soldiers who
before enlistment never knew what it was to have regular meals and
wear clean clothes every day, leave the service after a few years much
better citizens, and possessing a better education, for schools are
maintained in connection with all the barracks where instruction is given
in reading, writing and mathematics. The pay is about forty cents per
day, in Mexican silver, and is good pay for that country when you take
into consideration the fact that the soldier has absolutely no expenses
except for such luxuries as he may want.
The standing army of Mexico consists of thirty thousand men and
three thousand two hundred officers. Of this number the infantry
number twenty-two thousand six hundred, cavalry five thousand five
hundred, artillery two thousand, engineers and other branches of the
service making up the remainder. This gives a soldier for every five
hundred inhabitants, as compared with one for every fifteen hundred
inhabitants in the United States. Both infantry and cavalry are equipped
with the Spanish Mauser rifles and carbines. The headquarters of the
army are in the City of Mexico, and several battalions of infantry and
regiments of cavalry are stationed there at all times. The country is
divided into a number of districts, at the headquarters of each of which
are stationed large bodies of troops. Nearly every town of any size has a
commandancia where a few troops are quartered. This general
distribution of the military forces has been made with a prudent
foresight in order to prevent any local uprising.
 ARMY HEADQUARTERS, CITY OF MEXICO

In addition to the regular standing army, there are a number of armed

forces which would swell the number of available troops in time of war.
First and foremost are the Rurales who number about three thousand
five hundred by actual count, but double that number in effectiveness.
The Fiscal Guards number about one thousand and are in the revenue
service. The police of the states and cities are compelled to undergo
military drill also, and could be drafted into the army as trained soldiers.
These several forces would constitute another army almost equal in
number to the regular standing army. Militia organizations have been
provided for by law similar to those organizations in our own country,
but as yet little has been done. When these plans are perfected, it is
designed to have the total war footing number a force of one hundred
and fifty thousand drilled and disciplined men.
The President of Mexico is the commander-in-chief of the army and
navy. The “West Point of Mexico” is located next to the presidential
residence and is called the Chapultepec Military Academy. It was
founded in 1824. During the war of 1847 Chapultepec was successfully
stormed by the American forces, but heroically defended by the cadets.
A monument now stands at the foot of the hill in memory of those
cadets who fell in that engagement, and a graceful tribute is paid to the
memory of those youthful patriots on each fourth of July, when wreaths
are placed on the monument by the American residents of the capital at
the same time that they decorate the graves of American soldiers who
are buried near the city. This school now ranks high as a military school,
and more than one-third of the commissioned officers of the army are
graduates of this institution. The graduate leaves this school with the
rank of lieutenant. The student must bind himself to serve seven years
in the army, if he takes the technical courses, and, if he is discharged, or
refuses to serve, must repay to the government $16 for each month he
remained in the academy. If war should occur, all retired graduates
would be compelled to report for service.
Not a generation ago the capital itself was the home of innumerable
thieves. In fact, a goodly percentage of the people were either thieves,
robbers or beggars. These were drawn from the mestizo class, and
formed a picturesque but filthy group of blackguards. They would make
love to any one’s pocket, and argue with one another at the point of a
long, sharp knife. Each one carried a knife and revolver. “Unfortunate
men, women and children, the legitimate heritage of wrong, oppression
and misgovernment, thronged the streets begging in daytime, and
committing petty robberies by night. They slept by hundreds in
doorways, on benches in public parks, in ruined houses, and in the
dirtiest of apartments. A score or more of filthy beings of all ages and
both sexes would sleep together in one small room reeking with the
miasma that rose from sewers and unclean cobble-stone pavements.”
Vice was the natural outcome of such conditions. All natural feelings
of delicacy and shame were deadened. Morality was unknown, and they
lived like animals rather than human beings. Marriages were unthought
of, and children knew not their parents, for even their mothers deserted
them. If not deserted, they were frequently maimed and turned out into
the street to beg. Pulque and mescal added its touch to the picture.
Disfiguring diseases were added, and the name leperos attached to
them. Brantz Mayer, a writer on Mexico, has given the following
definition of the lepero. “Blacken a man in the sun, let his hair grow long
and tangled and become filled with vermin; let him plod about the
streets in all kinds of dirt for years, and never know the use of a brush,
or towel, or water, even, except in storms; let him put on a pair of
leather breeches at twenty and wear them until forty without change or
ablution; and over all place a torn and blackened hat and a tattered
blanket begrimed with abominations; let him have wild eyes and shining
teeth, features pinched by famine into sharpness, and breast bared and
browned; combine all these in your imagination and you have a recipe
for a Mexican lepero.”
These leperos were the thieving class. They frequented all parts of the
city. Even the churches were not exempt and you were just as likely to
be robbed by some apparently devout, kneeling worshipper saying his
ave marias in a sacred edifice as on the street. In the less frequented
streets many hold-ups took place, and the bodies of those murdered
would be found on the pavement in the morning. It was hardly safe to
move about the street after night had fallen. The thieves’ market was
well known and did a thriving business. Here were the pawn-brokers
who did a profitable business acting as “fences” for the thieves. Many
instances are told by foreigners who were robbed, and, in a few hours,
found their property exposed for sale in this market. They were obliged
to pay considerable sums to recover their own property.
All these types are now disappearing, and even the beggars are less
numerous. The former lawless leperos are now seen in the poor venders
of lottery tickets who crowd every public place. Begging is forbidden in
most parts of the city. Vice has not disappeared, it is true, nor has it in
American cities. The poor peon still gets intoxicated and is dirty, but he
is more law-abiding than formerly. Conditions, which are the result of
neglect and misrule of centuries, can only be overcome entirely by
education, immigration and the infusion of saner ideas, and this is a
gradual process. A whole city, or a whole country, can not be plowed up
and re-sown in a season, as the corn fields of last year were
transformed into the waving fields of golden grain this year. A
generation is even too short a time. The change actually wrought has
been almost a miracle. Work can now be had by all who are willing to
work, and the government is making strenuous efforts to get rid of the
idle classes. It is a long and hard task, but another decade under
present conditions will work wonders.
 An excellent police system is found in the capital and all the other
cities. A policeman is not hard to find. One is stationed at nearly every
street intersection. During the day he stands like a statue, occasionally
leaning against a door post. At night the policeman brings a lantern and
a blanket, and sets the lantern in the centre of the crossing, while he
stands beside it or not far away. The joker says the lantern is intended
to aid the thief in avoiding the officer of the law. Sometimes after the
people quit passing, he may lean up against a building and fall asleep,
but you can locate his vicinity by the lantern. As the windows are all
heavily barred, and the doors are heavy oaken affairs that it would take
a stick of dynamite to move, and as fires are infrequent, his lot is not a
very hard one. The police are very numerous, however, because the
government wants to keep informed in order that a revolutionary
movement may not gain any headway. One seldom hears of knock-
downs now, and pocket picking is about the only kind of robbery.
These guardians of the peace are generally called serenos. This name
clings to them from the old Spanish watchman whose duty it was to call
out the time of the night and state of the weather. As this was usually
clear, the watchman would say “tiempo sereno” meaning “weather
clear.” From the frequent repetition of this term the watchmen were
dubbed serenos. The Mexican sereno is generally a faithful and reliable
official and is obliging to a stranger. They have made the streets in the
City of Mexico as safe as in Paris. The senses of sight and smell may be
offended more often, but purse and life are just as secure.
 CHAPTER XIX
THE STORY OF THE REPUBLIC

There is a strange fascination about the history of Mexico, and no one

can thoroughly understand the country or the people without a little
insight into those stirring events that preceded the establishment of the
present republic. With the increasing friendly regard and the growing
commercial intercourse between the two countries, a few pages devoted
to this subject will not be amiss; and the prospective traveller, as well as
the one who has already travelled in that country, will find an additional
interest in Mexico and the Mexicans.
However we may feel inclined to criticize Cortez, the fact remains that
he thoroughly subjugated the country, and presented to Spain the
fairest jewel of her domain. Having been made the first governor of New
Spain, he was too busy with fresh conquests and the task of keeping
order to make a successful ruler. In order to reform the various abuses
that had grown up, and represent in every way possible the person of
the king, King Charles V sent the first viceroy in the autumn of 1535.
This first of a long line of viceroys, reaching down to the year 1821, was
named Antonio de Mendoza, himself of noble descent, a man of ability,
and one who had at heart the best interests of the colonists and the
welfare of the Indians. The latter had been subjected to many
humiliations and hardships all of which were removed by him, and they
were encouraged in the cultivation of the lands.
The colonists themselves were a source of great trouble for they were
mostly adventurers and were not, like the early American colonists, men
who were seeking religious liberty. The arm of the church was stretched
just as strongly in new Spain as in the land of their birth, and to the
religious orders was due in great measure the firm foundation upon
which the Spanish government was established. During the rule of this
man and his successor, Velasco, the country prospered, agriculture was
stimulated and a number of industries suitable to the climate of the
country encouraged.
 At the close of the sixteenth century, Spain underwent great changes.
The line of able rulers had passed away, and the government fell into
the hands of profligates who were favourites of the reigning sovereign.
The line of viceroys continued in rotation, and most of them were fair
men who probably governed the best they knew how, but their
knowledge on that subject was not very great. They were poor rulers
when compared with the first two above mentioned. The church
retained its firm grasp. As one writer has put it, during the first century
of Spanish rule the church was a blessing to the country, during the
second an indifferent quantity and during the third an actual menace.
The inquisition—that terrible institution—had been established in Mexico
as early as 1570. The first auto-da-fé was celebrated in 1574, when
“there perished twenty-one pestilent Lutherans.” Indians were exempt
from this institution and it was only aimed at heretics of other nations.
Large numbers were burned in the capital and other cities. In Puebla,
the old house of the inquisition was remodelled within the last half-
century, and a number of walled-up cells opened in which skeletons
were found—no doubt remains of victims who had been buried alive.
The inquisition was not formally abolished until the beginning of the last
century, just prior to the beginning of the movement for independence.
Even this concession, and the promise of correcting other abuses, did
not stop the growing discontent, for generations had grown up who had
few ties linking them to the mother country; who had intermarried with
native races; and who would be satisfied with nothing but complete
severance of their relations.
The beginning of the nineteenth century opened with a feeling of
unrest in all European nations and their colonies. When Napoleon
overturned monarchies, the idea of the divine right of kings received a
shock. Among the countries thus affected was Spain, which had dropped
down from the high pedestal it had formerly occupied. The eyes of the
people of Mexico were opened by the events in Europe, and also by the
successful revolution of the American colonies. All the offices of profit in
Mexico were held by Spaniards, and the policy of the mother country
toward her dependents was well expressed by one of the viceroys as
follows: “Let the people of these dominions learn once for all that they
were born to be silent and to obey, and not to discuss nor to have
opinions in political affairs.” The spirit of revolution and liberty was in the
air and restraint became more and more galling. The events leading to
the independence of Mexico, and the stirring times subsequent thereto,
can best be treated by a glance at the men who were in the limelight
during the various periods.
When Miguel Hidalgo, curate of the little village of Dolores, sounded
the “grito” of independence by ringing the bell of the parish church early
on the morning of the 16th of September, 1810, a struggle for
independence was started that lasted for eleven years, and during which
much of the soil of Mexico was crimsoned with the blood of those slain
in battle or executed by the authorities as traitors. At the outset no
people were less prepared for such a contest. They knew nothing of
military tactics; their weapons were primitive and their leaders were
without military training. No more righteous cause ever existed for
rebellion against tyranny and usurpation. The first two leaders were
consecrated representatives of the church that had assisted a despotic
government in bringing about such an unfortunate state of affairs.
These two martyrs who were excommunicated by the church, and
executed by the government as traitors, are now honoured with resting
places in sacred ground by a grateful nation.
The first revolt was headed by a picture of the patron saint of the
country, and shouts of “Viva Nuestra Señora de Guadalupe” and “Viva la
Independencia” were intermingled. Hidalgo and his compatriots were
compelled to begin their movement before thoroughly prepared,
because their plans had been discovered and betrayed to the
government. On the morning of the memorable day above mentioned,
Hidalgo addressed the people from the pulpit of the church where he
had so often celebrated mass, and, leading his followers forth, released
the prisoners in the town, and captured the principal Spaniards. Soon
afterwards this priest-warrior patriot, who had been named Captain-
General, followed by a few hundred of human beings (they can not be
called soldiers), marched forth to conquer Mexico and give “death to the
Spaniards.”
It was a motley crowd armed with stones, lances, machetes, arrows,
clubs and swords, whose numbers and enthusiasm were ever increasing
as they marched across the country without meeting resistance. San
Miguel and Celaya, Irapuato and Querétero, yielded, and the army which
by this time numbered tens of thousands marched towards Guanajuato.
The governor of that province assembled the terror-stricken populace in
the now famous Alhondiga de Granaditas, built for storing grain but now
a prison, as noted in that city as the Bastille of Paris. Upon a refusal to
surrender, Hidalgo and his followers attacked this fortress with fanatical
zeal, and captured it by the mere force of numbers. This supplied him
with plenty of food and a million dollars in money which furnished the
sinews of war.
Terror struck the hearts of the Spaniards and every town yielded to
this new leader, who now bore the title of Generalissimo, as the army
approached the City of Mexico. One terrible battle occurred at Monte las
Cruces and both forces withdrew. Hidalgo—and this was probably his
greatest error—retreated, and his fortune immediately turned. The
volatile nature of the people asserted itself and his followers deserted by
the thousands. He started for the United States, but was betrayed and
captured, and was executed at Chihuahua on July 31st, 1811. For ten
years his head was suspended by a spike from one of the corners of the
Alhondiga de Granaditas, once occupied by him as conqueror, as a
warning to revolutionists, but was afterward buried with great ceremony
in the cathedral at the capital.
It was around a disciple of Hidalgo that the forces of discontent and
patriotism rallied upon the death of their first leader, and that man was
also a priest, Jose Maria Morelas. Of low birth and poor, this man drove
mules until thirty years of age before an opportunity presented itself for
education to fit himself for the priesthood, which was his ambition. In
that time he had acquired the qualities of patience and cool calculation
from the animals he drove. A student under Hidalgo, he had imbibed a
love for independence, and leaving his church upon the sounding of the
“grito,” offered his services to the Generalissimo. He was an abler leader
than Hidalgo and showed great military skill, winning a series of victories
clear across the country from Acapulco, on the Pacific Coast, to Cuautla
not far from Vera Cruz. At Cuautla he was besieged for over two
months, and then successfully withdrew with all his forces by night.
Returning to Acapulco he summoned the first Mexican Congress, which
met at Chilpantzingo, a small town near that city. This congress met on
the 14th of September, 1813, and on the following day issued its famous
declaration of independence, as follows:—“The Congress of Anahuac,
lawfully installed in the city of Chilpantzingo, of North America, solemnly
declares, in the presence of God, arbitrator of kingdoms and author of
society, who gives and takes away according to the inscrutable designs
of his providence, that, through the present circumstances of Europe, it
has recovered the exercise of its sovereignty, hitherto usurped, its
dependence upon the throne of Spain being thus forever disrupted and
dissolved.”[5]
This congress provided a form of government with a military executive
called Generalissimo, and Morelas was elected to this position for life, or
“so long as he was worthy.” Shortly after this his forces were defeated at
Valladolid, now called Morelia, and his power began to wane, though
resistance was kept up for some time afterwards with varying success.
Spanish troops had arrived, and stronger leaders were in charge of the
government forces and the cause of independence looked dark. The
plans of Morelas were betrayed to the enemy and he was captured. The
ecclesiastical tribunes covered him with ignominy. He was then
sentenced to death by the military authorities, and shot in the little
village of San Cristobal Ecatepec, near the capital, on December 22d,
1815, dying the death of a hero. This muleteer-priest-warrior was an
able leader, an honest man and a patriotic citizen. He seemed devoid of
personal ambition, although accepting title for the sake of the cause he
fought for. He was possessed of restless energy and great piety, for he
always made confession before entering battle. To-day, he is second
only to Hidalgo in the affections of the people, and worthily fills that
position. Over the door of the house once owned by him in Morelia
appears the following inscription:—
“Morelas the illustrious
Immortal Hero.
In this house honoured by thy presence
Salute you the grateful people of Morelia.”
The revolution was seemingly crushed at the death of Morelas but a
few patriots retired to the mountains, and there kept alive for better
days the sacred fire of liberty. Guerrero was one of those heroes who
showed an unwearying activity, and kept up a constant warfare upon
the government forces. The next prominent name in succession among
those leaders of the movement for freedom was Agustin de Iturbide, a
former active and able officer of the royalist forces, and to whom more
than anyone else was due the failure of Morelas. Deserting the cause of
Spain, because he thought injustice had been done him, General
Iturbide issued the “Plan of Iguala” on the 20th of February, 1820,
composed of three articles: preservation of the Roman Catholic church;
independence of Mexico under a monarchical form of government with a
prince of the royal house of Spain as ruler; union and equality of
Spaniards and Mexicans. From this proclamation his army became
known as the army of the three guarantees. His act was full of duplicity,
for he had obtained the largest force possible from the Viceroy Apodaca
in order to turn them over to the new scheme.
Before the viceroy could recover from his surprise, Iturbide, who had
been joined by most of the insurgent leaders, had started on his
victorious campaign. Valladolid, Querétero and Puebla succumbed. The
viceroy tried by suppressing liberty, and enforcing enlistments in the
royal army, to stem the tide but in vain, and he was deposed. O’Donoju,
the sixty-fourth and last viceroy, arrived about this time at Vera Cruz,
but was intercepted by Iturbide and entered into the treaty of Cordoba
in which the independence of Mexico was recognized with a sovereign to
be selected from the royal house of Spain, and a provisional Junta
formed. Iturbide was selected as president of this Junta, and made a
triumphal entry into the City of Mexico on the 27th of September, 1821.
This ended three hundred years of Spanish rule in Mexico. Iturbide had
accomplished in a little more than a year, and with little bloodshed, what
ten years of strife had failed to do. He can not be classed with Hidalgo
and Morelas as a pure patriot, but he has been officially designated as
the “Liberator of Mexico.”
The rejection of the treaty of Cordoba by the Cortes of Spain gave
new impetus to the smouldering ambitions of Iturbide. The second
Mexican Congress having been called, Iturbide at a packed session was
declared Emperor by a majority of four to one of those voting, but not a
constitutional majority, and he took the office as Agustin I. When he was
crowned and anointed in the cathedral with much form and solemnity,
on the 21st of July, 1822, the ambition of this self-made emperor had
reached its full. The saying that uneasy lies the head that wears a crown
never had better application than in this instance. Other leaders in the
cause of liberty felt that they had been slighted, and every discontented
person made common cause against the Emperor. A republic was
proclaimed at Vera Cruz in December of the same year by Santa Anna,
who was commander of a regiment stationed there, and he issued a
pronunciamento. This plan failed, but it encouraged Bravo, Guerrero and
other revolutionary leaders, and rebellion sprung up in a number of
places. Iturbide had dissolved congress and this increased the
dissatisfaction. A more formidable revolt arose, and on March 19th,
1823, Iturbide abdicated without attempting to retain his position by
force of arms.
A few weeks later the ex-Emperor left Mexico and sailed for Italy,
having been granted an annual sum of $25,000 for his services. He soon
went to England and wrote the government from there that the republic
was in danger, and he would come back to help fight the battles of his
country. He did not know that his death had been decreed by congress,
and so he set sail upon his last voyage. When he arrived at Vera Cruz he
was captured, and after some delay was executed at Padilla on the 19th
of July, 1824, as a traitor, in his forty-first year. His body was buried in a
roofless old church and lay there until 1838, when it was removed to the
Cathedral.
Opinion is very much divided as to the rank that should be accorded
Iturbide. He was able, brave, honest so far as is known, and probably
fell a victim to his ambition like many a man before him. The relative
regard in which he is held is shown in the fact that the town which gave
both him and his former vanquished foe, Morelas, birth, is now called
Morelia, and a state is also named Morelas. In contrast to this there is
neither city nor state named after Iturbide, and the famous Iturbide
Hotel in the capital city, once his residence, is the only institution
perpetuating his name so far as I could learn. The only things
accomplished by him during his brief reign were the settlement of the
titles by which he and his family should be addressed, the succession to
the throne, order of precedence among the dignitaries, allowances of
himself and family, and the creation of the Order of Guadalupe to
bestow honours upon his followers.
At last a so-called republic was established, and Guadalupe Victoria
was inaugurated as the first president on the 10th day of October, 1824,
and served until 1828. When the fort of San Juan de Ulua at Vera Cruz
lowered its flag, in 1825, the last vestige of Spanish power was gone,
and the red and yellow striped banner of the Iberian peninsula was not
to be seen on Mexican soil. And Mexico, as then constituted, was a big
country, containing almost twice as much territory as to-day. From the
end of the administration of President Victoria until after the death of
Maximilian, there was not a year of peace in Mexico. Revolutions,
pronunciamentos, “plans” and restorations followed each other in quick
succession. Generals, presidents and dictators sprang up like
mushrooms, and their position was as evanescent. The congress
unwisely decreed the expulsion of the Spaniards, and their departure
took much of the wealth of the country. Revolutions were an every-day
affair. A man in position of authority did not know when his time to be
shot might come. A sudden turn of fortune might send him either to the
national palace, or before a squad of men with guns aimed at his heart.
A good illustration of this uncertainty of affairs is seen in the
treatment and fate of the grim old patriot Guerrero. Born of very low
Indian parents he had climbed to the front and borne many of the
burdens of the struggle with Spain. He cheerfully yielded his command
to the renegade Iturbide, and fought valiantly under that leader for
liberty. By a turn of fortune he became the third president in 1829. A
few months later he was compelled to flee, but was soon afterwards
betrayed and captured at Acapulco. At a farcical trial he was condemned
to death as “morally incapable” to act as president, and shot on the 15th
of February, 1831, at Cuilapa. Soon afterwards he was declared a martyr
and his body removed to the capital with honours. Two monuments to
this martyr now adorn that city, and a state has been named after him.
Under his short rule slavery was abolished by statute.
Elections eventually became a farce. The unfortunate habit was
acquired of appealing to arms instead of submitting to the result of the
ballot. The trouble was that the people had copied the letter, and not
the spirit of the American constitution. Liberty was interpreted as
license, after their exaggerated ideas of the former. The scheming
politicians would hesitate at nothing—revolution or civil war—to attain
private ends or personal aggrandizement. A general indolence of
character, and the hindrances to the acquirement of property among the
masses, made the people more willing to yield to disturbing and
designing politicians. They are impetuous by nature, impatient of
restraint and easily fired up. The rapid changes in government can be
seen when you read that there were five different presidents in each of
the years 1846 and 1847, and four in 1855—not an evidence of
tranquillity at least. The two leading parties constantly at war were the
“progresistas” and “retrogrados.”
During this period a few prominent names are constantly recurring,
and by far the most prominent one is that of the notorious Santa Anna,
who, for more than fifty years, occupied a prominent, but not always
honourable, place in Mexican affairs. Earlier in life his restless energy
was expended in a fairly commendable way, and he fought some battles
in defense of the rights of the people. During the war of intervention
with France in 1838 he lost his leg in the defense of Vera Cruz. Ever
afterwards, when in trouble, he would flourish his severed limb and
remind the people how he had been mutilated in the defense of his
country, with the effect of restoring himself in public favour. As he grew
older his naturally quarrelsome disposition increased, his vanity knew no
bounds, and when at the height of his glory, he declared himself dictator
and ordered all people to address him as “most serene highness.” Never
honest except as a matter of policy, his cupidity became more
pronounced, until, near the close of the war with the United States, he
offered to appoint commissioners and confirm a treaty of peace for the
sum of one million dollars. First elected president in 1833, he was again
either chosen to, or assumed the office, in 1839, 1846, 1847, 1853 and
1855, but did not serve long at any time. On one occasion his
amputated leg was buried with great ceremony, but afterwards fickle
sentiment changed, and the martyr part of this hero was brought forth
by the rabble, dragged through the streets of the capital, and insulting
epithets heaped upon the former idol.
Santa Anna led the forces against the Texas insurrectionists, and was
the man responsible for the Alamo slaughter, when one hundred and
forty brave Texans were trapped and slain. Visitors to that place are still
shown the stains made by the blood of that brave frontiersman, Davy
Crockett, and the cry of “Remember the Alamo” still has potency. This
insurrection was soon followed by the war between Mexico and the
United States.
 Franklin says, there never was a good war nor a bad peace. The
United States can not be justified in warring upon Mexico, though the
results have perhaps been for the best with both nations. Bancroft does
not mince words in his treatment of the subject for he says: “It (the
Mexican War) was a premeditated and predetermined affair; it was the
result of a deliberately calculated scheme of robbery on the part of the
superior force.” The result was a foregone conclusion, for Mexico, torn
by internal dissensions, impoverished by the expense of revolutions and
official robbery, and with a government changing with every change of
the seasons, had neither armies, money nor supplies for such a conflict.
The people were used to the smell of powder but were not trained
soldiers, and the “generals” were simply a few of the twelve thousand
recipients of military commissions that had been distributed by various
presidents in the preceding three years. “Plans” promulgated by one
party were bombarded with “pronunciamentos” from another. This was
the condition of affairs when General Taylor assumed the offensive and
fought the battle of Palo Alto.
Mexico might have sued for peace at this time, but no government
was in power long enough to negotiate a treaty. A special envoy sent
from Washington at the request of one president was refused an
audience by a new one, who had usurped the office before his arrival.
Generals Taylor and Fremont subdued Northern Mexico, and General
Scott later began his memorable march toward the ancient Aztec capital,
from Vera Cruz, like Cortez of old. Santa Anna, who had been
“recuperating” from public unpopularity at Havana, returned and state
after state immediately “pronounced” in his favour. He issued a
manifesto assuming the executive control and took the field against the
invaders. He first tried to secure $15,000,000 from the Church, but
although the priests hated the “northern heretics” they were loth to give
up the coin, and little was secured. Vera Cruz fell after two weeks’
bombardment, and Puebla yielded to the Americans. Patriotism was
finally aroused to save the City of Mexico, but the victories of
Chapultepec, Chorubusco and Molino del Rey were followed by the
triumphal entry of General Scott into the capital. The treaty of
Guadalupe-Hidalgo ceded to the United States more than six hundred
thousand square miles of the Mexican domain, including some of the
richest mineral lands of the republic. Disgraced and humiliated as Mexico
had been, it was, I believe, the beginning of better things for that
country.
Santa Anna went into voluntary exile to Jamaica. The first president
after the war, Herrera, actually served the appointed time of his office,
but disorder soon began under his successor. “Pronouncing” became
popular again, and Santa Anna returned. He was made dictator for a
short time by his favourites. This was the last office held by this selfish
politician. He exiled himself to St. Thomas again, and afterwards in
Elizabethport, New Jersey. During the second empire he tried to curry
favour with both sides, but neither would listen to him. Discouraged and
disheartened he lived abroad, until, burdened by the weight of eighty
years, he sought and obtained permission to return to the capital, and
died on the 20th of June, 1876. Thus passed a man who had lived in
stirring times, was most intensely hated, had been president six times,
military dictator four times, had upset fifteen governments, had been
marked for the assassin’s bullet many times; and yet he lived to a ripe
old age and died a natural death. However, all his glory had faded, and,
blind, lame and infirm, he spent his last days in extreme poverty.
Here is a picture of this man drawn by Rev. William Butler,[6] who
visited him about a year before his death: “Santa Anna was living in an
obscure street, neglected and forgotten by all parties. On entering the
apartment we found the old man sitting on a sofa, behind which hung a
picture of his wife ‘her serene highness, Dolores Tosta de Santa Anna’
arrayed as a vice-queen. The magnificence of the painting contrasted
sadly with the poverty-stricken aspect of the room and furniture. To him,
however, this could make but little difference, as we soon saw that he
was totally blind as well as feeble and broken in spirit, with a tendency
to mental weakness.” He was buried in the cemetery at Guadalupe
without honours or recognition by the government, and his remains still
rest there. As I gazed upon his tomb I could not help thinking what a
contrast between his career and that of the patriots Hidalgo, Morelas,
and Juarez.
The early constitution had declared that the Roman Catholic religion
should perpetually be the religion of Mexico. Nevertheless a struggle had
been growing up between the clericals and liberals for many years with
increasing intensity. It finally centred in a struggle over the