Visit https://ebookgate.

com to download the full version and

explore more ebooks

Multilinear Subspace Learning Dimensionality

Reduction of Multidimensional Data 1st Edition
Haiping Lu

_____ Click the link below to download _____

https://ebookgate.com/product/multilinear-subspace-
learning-dimensionality-reduction-of-multidimensional-
data-1st-edition-haiping-lu/

Explore and download more ebooks at ebookgate.com

Here are some recommended products that might interest you.
You can download now and explore!

Geometric Structure of High Dimensional Data and

Dimensionality Reduction English Version Chinese Edition
Wang Jian Zhong
https://ebookgate.com/product/geometric-structure-of-high-dimensional-
data-and-dimensionality-reduction-english-version-chinese-edition-
wang-jian-zhong/
ebookgate.com

Hipparcos the New Reduction of the Raw Data Astrophysics

and Space Science Library 1st Edition Floor Van Leeuwen

https://ebookgate.com/product/hipparcos-the-new-reduction-of-the-raw-
data-astrophysics-and-space-science-library-1st-edition-floor-van-
leeuwen/
ebookgate.com

Learning Qlikview Data Visualization 1st Edition Karl

Pover

https://ebookgate.com/product/learning-qlikview-data-
visualization-1st-edition-karl-pover/

ebookgate.com

Scientific Inference Learning From Data 1st Edition Simon

Vaughan

https://ebookgate.com/product/scientific-inference-learning-from-
data-1st-edition-simon-vaughan/

ebookgate.com
Data Mining and Machine Learning in Cybersecurity 1st
Edition Sumeet Dua

https://ebookgate.com/product/data-mining-and-machine-learning-in-
cybersecurity-1st-edition-sumeet-dua/

ebookgate.com

Multidimensional Poverty Measurement and Analysis 1st

Edition Sabina Alkire

https://ebookgate.com/product/multidimensional-poverty-measurement-
and-analysis-1st-edition-sabina-alkire/

ebookgate.com

Multidimensional Real Analysis II Integration 1st Edition

J.J. Duistermaat

https://ebookgate.com/product/multidimensional-real-analysis-ii-
integration-1st-edition-j-j-duistermaat/

ebookgate.com

Unsaturated Soil Mechanics 1st Edition Ning Lu

https://ebookgate.com/product/unsaturated-soil-mechanics-1st-edition-
ning-lu/

ebookgate.com

Epigenetics and Dermatology 1st Edition Qianjin Lu

https://ebookgate.com/product/epigenetics-and-dermatology-1st-edition-
qianjin-lu/

ebookgate.com
 Chapman & Hall/CRC
Machine Learning & Pattern Recognition Series

Multilinear
Subspace Learning
Dimensionality Reduction of
Multidimensional Data

Haiping Lu
Konstantinos N. Plataniotis
Anastasios N. Venetsanopoulos
 Multilinear
Subspace Learning
Dimensionality Reduction of
Multidimensional Data
 Chapman & Hall/CRC
Machine Learning & Pattern Recognition Series

SERIES EDITORS

Ralf Herbrich Thore Graepel

Amazon Development Center Microsoft Research Ltd.
Berlin, Germany Cambridge, UK

AIMS AND SCOPE

This series reÀects the latest advances and applications in machine learning and pat-
tern recognition through the publication of a broad range of reference works, text-
books, and handbooks. The inclusion of concrete examples, applications, and meth-
ods is highly encouraged. The scope of the series includes, but is not limited to, titles
in the areas of machine learning, pattern recognition, computational intelligence,
robotics, computational/statistical learning theory, natural language processing,
computer vision, game AI, game theory, neural networks, computational neurosci-
ence, and other relevant topics, such as machine learning applied to bioinformatics
or cognitive science, which might be proposed by potential contributors.

PUBLISHED TITLES

MACHINE LEARNING: An Algorithmic Perspective

Stephen Marsland
HANDBOOK OF NATURAL LANGUAGE PROCESSING,
Second Edition
Nitin Indurkhya and Fred J. Damerau
UTILITY-BASED LEARNING FROM DATA
Craig Friedman and Sven Sandow
A FIRST COURSE IN MACHINE LEARNING
Simon Rogers and Mark Girolami
COST-SENSITIVE MACHINE LEARNING
Balaji Krishnapuram, Shipeng Yu, and Bharat Rao
ENSEMBLE METHODS: FOUNDATIONS AND ALGORITHMS
Zhi-Hua Zhou
MULTI-LABEL DIMENSIONALITY REDUCTION
Liang Sun, Shuiwang Ji, and Jieping Ye
BAYESIAN PROGRAMMING
Pierre Bessière, Emmanuel Mazer, Juan-Manuel Ahuactzin, and Kamel Mekhnacha
MULTILINEAR SUBSPACE LEARNING: DIMENSIONALITY REDUCTION
OF MULTIDIMENSIONAL DATA
Haiping Lu, Konstantinos N. Plataniotis, and Anastasios N. Venetsanopoulos
 Chapman & Hall/CRC
Machine Learning & Pattern Recognition Series

Multilinear
Subspace Learning
Dimensionality Reduction of
Multidimensional Data

Haiping Lu
Konstantinos N. Plataniotis
Anastasios N. Venetsanopoulos
MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does
not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MAT-
LAB® software or related products does not constitute endorsement or sponsorship by The MathWorks
of a particular pedagogical approach or particular use of the MATLAB® software.

CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2014 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Version Date: 20131021

International Standard Book Number-13: 978-1-4398-5729-8 (eBook - PDF)

This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.copy-
right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222
Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro-
vides licenses and registration for a variety of users. For organizations that have been granted a pho-
tocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To Hongxia, Dailian, and Daizhen

To Ilda
Contents

List of Figures xiii

List of Tables xvii

List of Algorithms xix

Acronyms and Symbols xxi

Preface xxv

1 Introduction 1
1.1 Tensor Representation of Multidimensional Data . . . . . . . 2
1.2 Dimensionality Reduction via Subspace Learning . . . . . . . 5
1.3 Multilinear Mapping for Subspace Learning . . . . . . . . . . 9
1.4 Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

I Fundamentals and Foundations 17

2 Linear Subspace Learning for Dimensionality Reduction 19
2.1 Principal Component Analysis . . . . . . . . . . . . . . . . . 20
2.2 Independent Component Analysis . . . . . . . . . . . . . . . 24
2.3 Linear Discriminant Analysis . . . . . . . . . . . . . . . . . . 27
2.4 Canonical Correlation Analysis . . . . . . . . . . . . . . . . . 31
2.5 Partial Least Squares Analysis . . . . . . . . . . . . . . . . . 35
2.6 Unified View of PCA, LDA, CCA, and PLS . . . . . . . . . 39
2.7 Regularization and Model Selection . . . . . . . . . . . . . . 40
2.7.1 Regularizing Covariance Matrix Estimation . . . . . . 40
2.7.2 Regularizing Model Complexity . . . . . . . . . . . . . 41
2.7.3 Model Selection . . . . . . . . . . . . . . . . . . . . . . 42
2.8 Ensemble Learning . . . . . . . . . . . . . . . . . . . . . . . 43
2.8.1 Bagging . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.8.2 Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.10 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 46

vii
viii Contents

3 Fundamentals of Multilinear Subspace Learning 49

3.1 Multilinear Algebra Preliminaries . . . . . . . . . . . . . . . 50
3.1.1 Notations and Definitions . . . . . . . . . . . . . . . . 50
3.1.2 Basic Operations . . . . . . . . . . . . . . . . . . . . . 53
3.1.3 Tensor/Matrix Distance Measure . . . . . . . . . . . . 56
3.2 Tensor Decompositions . . . . . . . . . . . . . . . . . . . . . 57
3.2.1 CANDECOMP/PARAFAC . . . . . . . . . . . . . . . 57
3.2.2 Tucker Decomposition and HOSVD . . . . . . . . . . 58
3.3 Multilinear Projections . . . . . . . . . . . . . . . . . . . . . 59
3.3.1 Vector-to-Vector Projection . . . . . . . . . . . . . . . 59
3.3.2 Tensor-to-Tensor Projection . . . . . . . . . . . . . . . 61
3.3.3 Tensor-to-Vector Projection . . . . . . . . . . . . . . . 61
3.4 Relationships among Multilinear Projections . . . . . . . . . 63
3.5 Scatter Measures for Tensors and Scalars . . . . . . . . . . . 64
3.5.1 Tensor-Based Scatters . . . . . . . . . . . . . . . . . . 64
3.5.2 Scalar-Based Scatters . . . . . . . . . . . . . . . . . . 67
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.7 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 69

4 Overview of Multilinear Subspace Learning 71

4.1 Multilinear Subspace Learning Framework . . . . . . . . . . 72
4.2 PCA-Based MSL Algorithms . . . . . . . . . . . . . . . . . . 74
4.2.1 PCA-Based MSL through TTP . . . . . . . . . . . . . 74
4.2.2 PCA-Based MSL through TVP . . . . . . . . . . . . . 76
4.3 LDA-Based MSL Algorithms . . . . . . . . . . . . . . . . . . 76
4.3.1 LDA-Based MSL through TTP . . . . . . . . . . . . . 77
4.3.2 LDA-Based MSL through TVP . . . . . . . . . . . . . 77
4.4 History and Related Works . . . . . . . . . . . . . . . . . . . 78
4.4.1 History of Tensor Decompositions . . . . . . . . . . . 78
4.4.2 Nonnegative Matrix and Tensor Factorizations . . . . 79
4.4.3 Tensor Multiple Factor Analysis and Multilinear Graph-
Embedding . . . . . . . . . . . . . . . . . . . . . . . . 80
4.5 Future Research on MSL . . . . . . . . . . . . . . . . . . . . 81
4.5.1 MSL Algorithm Development . . . . . . . . . . . . . . 81
4.5.2 MSL Application Exploration . . . . . . . . . . . . . . 84
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.7 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 86

5 Algorithmic and Computational Aspects 89

5.1 Alternating Partial Projections for MSL . . . . . . . . . . . . 90
5.2 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.2.1 Popular Initialization Methods . . . . . . . . . . . . . 92
5.2.2 Full Projection Truncation . . . . . . . . . . . . . . . 93
5.2.3 Interpretation of Mode-n Eigenvalues . . . . . . . . . 94
5.2.4 Analysis of Full Projection Truncation . . . . . . . . . 95
 Contents ix

5.3 Projection Order, Termination, and Convergence . . . . . . . 96

5.4 Synthetic Data for Analysis of MSL Algorithms . . . . . . . 97
5.5 Feature Selection for TTP-Based MSL . . . . . . . . . . . . 99
5.5.1 Supervised Feature Selection . . . . . . . . . . . . . . 100
5.5.2 Unsupervised Feature Selection . . . . . . . . . . . . . 101
5.6 Computational Aspects . . . . . . . . . . . . . . . . . . . . . 101
5.6.1 Memory Requirements and Storage Needs . . . . . . . 101
5.6.2 Computational Complexity . . . . . . . . . . . . . . . 102
5.6.3 MATLAB R
Implementation Tips for Large Datasets 102
5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.8 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 104

II Algorithms and Applications 105

6 Multilinear Principal Component Analysis 107
6.1 Generalized PCA . . . . . . . . . . . . . . . . . . . . . . . . 108
6.1.1 GPCA Problem Formulation . . . . . . . . . . . . . . 108
6.1.2 GPCA Algorithm Derivation . . . . . . . . . . . . . . 109
6.1.3 Discussions on GPCA . . . . . . . . . . . . . . . . . . 110
6.1.4 Reconstruction Error Minimization . . . . . . . . . . . 112
6.2 Multilinear PCA . . . . . . . . . . . . . . . . . . . . . . . . . 113
6.2.1 MPCA Problem Formulation . . . . . . . . . . . . . . 114
6.2.2 MPCA Algorithm Derivation . . . . . . . . . . . . . . 114
6.2.3 Discussions on MPCA . . . . . . . . . . . . . . . . . . 116
6.2.4 Subspace Dimension Determination . . . . . . . . . . 118
6.2.4.1 Sequential Mode Truncation . . . . . . . . . 119
6.2.4.2 Q-Based Method . . . . . . . . . . . . . . . . 119
6.3 Tensor Rank-One Decomposition . . . . . . . . . . . . . . . . 120
6.3.1 TROD Problem Formulation . . . . . . . . . . . . . . 120
6.3.2 Greedy Approach for TROD . . . . . . . . . . . . . . 121
6.3.3 Solving for the pth EMP . . . . . . . . . . . . . . . . . 122
6.4 Uncorrelated Multilinear PCA . . . . . . . . . . . . . . . . . 124
6.4.1 UMPCA Problem Formulation . . . . . . . . . . . . . 124
6.4.2 UMPCA Algorithm Derivation . . . . . . . . . . . . . 125
6.4.3 Discussions on UMPCA . . . . . . . . . . . . . . . . . 130
6.5 Boosting with MPCA . . . . . . . . . . . . . . . . . . . . . . 131
6.5.1 Benefits of MPCA-Based Booster . . . . . . . . . . . . 132
6.5.2 LDA-Style Boosting on MPCA Features . . . . . . . . 132
6.5.3 Modified LDA Learner . . . . . . . . . . . . . . . . . . 134
6.6 Other Multilinear PCA Extensions . . . . . . . . . . . . . . . 135
6.6.1 Two-Dimensional PCA . . . . . . . . . . . . . . . . . 135
6.6.2 Generalized Low Rank Approximation of Matrices . . 136
6.6.3 Concurrent Subspace Analysis . . . . . . . . . . . . . 136
6.6.4 MPCA plus LDA . . . . . . . . . . . . . . . . . . . . . 137
6.6.5 Nonnegative MPCA . . . . . . . . . . . . . . . . . . . 137
x Contents

6.6.6 Robust Versions of MPCA . . . . . . . . . . . . . . . . 137

6.6.7 Incremental Extensions of MPCA . . . . . . . . . . . . 138
6.6.8 Probabilistic Extensions of MPCA . . . . . . . . . . . 138
6.6.9 Weighted MPCA and MPCA for Binary Tensors . . . 139

7 Multilinear Discriminant Analysis 141

7.1 Two-Dimensional LDA . . . . . . . . . . . . . . . . . . . . . 142
7.1.1 2DLDA Problem Formulation . . . . . . . . . . . . . . 142
7.1.2 2DLDA Algorithm Derivation . . . . . . . . . . . . . . 143
7.2 Discriminant Analysis with Tensor Representation . . . . . . 145
7.2.1 DATER Problem Formulation . . . . . . . . . . . . . . 145
7.2.2 DATER Algorithm Derivation . . . . . . . . . . . . . 146
7.3 General Tensor Discriminant Analysis . . . . . . . . . . . . . 147
7.4 Tensor Rank-One Discriminant Analysis . . . . . . . . . . . 150
7.4.1 TR1DA Problem Formulation . . . . . . . . . . . . . . 150
7.4.2 Solving for the pth EMP . . . . . . . . . . . . . . . . . 151
7.5 Uncorrelated Multilinear Discriminant Analysis . . . . . . . 153
7.5.1 UMLDA Problem Formulation . . . . . . . . . . . . . 153
7.5.2 R-UMLDA Algorithm Derivation . . . . . . . . . . . . 154
7.5.3 Aggregation of R-UMLDA Learners . . . . . . . . . . 160
7.6 Other Multilinear Extensions of LDA . . . . . . . . . . . . . 162
7.6.1 Graph-Embedding for Dimensionality Reduction . . . 162
7.6.2 Graph-Embedding Extensions of Multilinear Discrimi-
nant Analysis . . . . . . . . . . . . . . . . . . . . . . . 163
7.6.3 Incremental and Sparse Multilinear Discriminant Anal-
ysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

8 Multilinear ICA, CCA, and PLS 165

8.1 Overview of Multilinear ICA Algorithms . . . . . . . . . . . 166
8.1.1 Multilinear Approaches for ICA on Vector-Valued Data 166
8.1.2 Multilinear Approaches for ICA on Tensor-Valued Data 166
8.2 Multilinear Modewise ICA . . . . . . . . . . . . . . . . . . . 167
8.2.1 Multilinear Mixing Model for Tensors . . . . . . . . . 168
8.2.2 Regularized Estimation of Mixing Tensor . . . . . . . 168
8.2.3 MMICA Algorithm Derivation . . . . . . . . . . . . . 169
8.2.4 Architectures and Discussions on MMICA . . . . . . . 170
8.2.5 Blind Source Separation on Synthetic Data . . . . . . 171
8.3 Overview of Multilinear CCA Algorithms . . . . . . . . . . . 172
8.4 Two-Dimensional CCA . . . . . . . . . . . . . . . . . . . . . 173
8.4.1 2D-CCA Problem Formulation . . . . . . . . . . . . . 173
8.4.2 2D-CCA Algorithm Derivation . . . . . . . . . . . . . 174
8.5 Multilinear CCA . . . . . . . . . . . . . . . . . . . . . . . . . 176
8.5.1 MCCA Problem Formulation . . . . . . . . . . . . . . 176
8.5.2 MCCA Algorithm Derivation . . . . . . . . . . . . . . 178
8.5.3 Discussions on MCCA . . . . . . . . . . . . . . . . . . 184
 Contents xi

8.6 Multilinear PLS Algorithms . . . . . . . . . . . . . . . . . . 184

8.6.1 N -Way PLS . . . . . . . . . . . . . . . . . . . . . . . . 184
8.6.2 Higher-Order PLS . . . . . . . . . . . . . . . . . . . . 185

9 Applications of Multilinear Subspace Learning 189

9.1 Pattern Recognition System . . . . . . . . . . . . . . . . . . 190
9.2 Face Recognition . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.2.1 Algorithms and Their Settings . . . . . . . . . . . . . 192
9.2.2 Recognition Results for Supervised Learning Algo-
rithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
9.2.3 Recognition Results for Unsupervised Learning Algo-
rithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
9.3 Gait Recognition . . . . . . . . . . . . . . . . . . . . . . . . . 196
9.4 Visual Content Analysis in Computer Vision . . . . . . . . . 198
9.4.1 Crowd Event Visualization and Clustering . . . . . . . 198
9.4.2 Target Tracking in Video . . . . . . . . . . . . . . . . 199
9.4.3 Action, Scene, and Object Recognition . . . . . . . . . 199
9.5 Brain Signal/Image Processing in Neuroscience . . . . . . . . 200
9.5.1 EEG Signal Analysis . . . . . . . . . . . . . . . . . . . 200
9.5.2 fMRI Image Analysis . . . . . . . . . . . . . . . . . . . 201
9.6 DNA Sequence Discovery in Bioinformatics . . . . . . . . . . 202
9.7 Music Genre Classification in Audio Signal Processing . . . . 202
9.8 Data Stream Monitoring in Data Mining . . . . . . . . . . . 203
9.9 Other MSL Applications . . . . . . . . . . . . . . . . . . . . 204

Appendix A Mathematical Background 205

A.1 Linear Algebra Preliminaries . . . . . . . . . . . . . . . . . . 205
A.1.1 Transpose . . . . . . . . . . . . . . . . . . . . . . . . . 205
A.1.2 Identity and Inverse Matrices . . . . . . . . . . . . . . 206
A.1.3 Linear Independence and Vector Space Basis . . . . . 206
A.1.4 Products of Vectors and Matrices . . . . . . . . . . . . 207
A.1.5 Vector and Matrix Norms . . . . . . . . . . . . . . . . 209
A.1.6 Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
A.1.7 Determinant . . . . . . . . . . . . . . . . . . . . . . . 210
A.1.8 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . 211
A.1.9 Generalized Eigenvalues and Eigenvectors . . . . . . . 212
A.1.10 Singular Value Decomposition . . . . . . . . . . . . . . 212
A.1.11 Power Method for Eigenvalue Computation . . . . . . 213
A.2 Basic Probability Theory . . . . . . . . . . . . . . . . . . . . 213
A.2.1 One Random Variable . . . . . . . . . . . . . . . . . . 213
A.2.2 Two Random Variables . . . . . . . . . . . . . . . . . 214
A.3 Basic Constrained Optimization . . . . . . . . . . . . . . . . 215
A.4 Basic Matrix Calculus . . . . . . . . . . . . . . . . . . . . . . 215
A.4.1 Basic Derivative Rules . . . . . . . . . . . . . . . . . . 215
A.4.2 Derivative of Scalar/Vector with Respect to Vector . . 216
xii Contents

A.4.3 Derivative of Trace with Respect to Matrix . . . . . . 216

A.4.4 Derivative of Determinant with Respect to Matrix . . 217

Appendix B Data and Preprocessing 219

B.1 Face Databases and Preprocessing . . . . . . . . . . . . . . . 219
B.1.1 PIE Database . . . . . . . . . . . . . . . . . . . . . . . 219
B.1.2 FERET Database . . . . . . . . . . . . . . . . . . . . 220
B.1.3 Preprocessing of Face Images for Recognition . . . . . 220
B.2 Gait Database and Preprocessing . . . . . . . . . . . . . . . 222
B.2.1 USF Gait Challenge Database . . . . . . . . . . . . . 222
B.2.2 Gait Silhouette Extraction . . . . . . . . . . . . . . . . 224
B.2.3 Normalization of Gait Samples . . . . . . . . . . . . . 224

Appendix C Software 227

C.1 Software for Multilinear Subspace Learning . . . . . . . . . . 227
C.2 Benefits of Open-Source Software . . . . . . . . . . . . . . . 228
C.3 Software Development Tips . . . . . . . . . . . . . . . . . . . 228

Bibliography 231

Index 263
List of Figures

1.1 Examples of second-order tensor (matrix) data. . . . . . . . 1

1.2 Examples of third-order tensor data. . . . . . . . . . . . . . 3
1.3 Examples of fourth-order tensor data. . . . . . . . . . . . . . 4
1.4 A third-order tensor formed by the Gabor filter outputs of a
gray-level face image. . . . . . . . . . . . . . . . . . . . . . . 5
1.5 A face can be represented as one point in an image space of
the same size. . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 Linear versus multilinear mapping. . . . . . . . . . . . . . . 7
1.7 Reshaping (vectorization) of a 32 × 32 face image to a 1024 ×
1 vector breaks the natural structure and correlation in the
original face image. . . . . . . . . . . . . . . . . . . . . . . . 7
1.8 Vector-based versus tensor-based analysis of a 3D object. . . 8
1.9 The field of matrix computations seems to “kick up” its level
of thinking about every 20 years. . . . . . . . . . . . . . . . 10
1.10 Illustration of second-order feature characteristics. . . . . . . 11
1.11 Illustration of third-order feature characteristics. . . . . . . 12

2.1 Linear subspace learning solves for a projection matrix U,

which maps a high-dimensional vector x to a low-dimensional
vector y. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.1 Multilinear subspace learning finds a lower-dimensional repre-

sentation by direct mapping of tensors through a multilinear
projection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2 Illustration of tensors of order N = 0, 1, 2, 3, 4. . . . . . . . . 50
3.3 Illustration of the mode-n vectors. . . . . . . . . . . . . . . . 52
3.4 Illustration of the mode-n slices. . . . . . . . . . . . . . . . . 52
3.5 An example of second-order rank-one tensor (that is, rank-one
T
matrix): A = u(1) ◦ u(2) = u(1) u(2) . . . . . . . . . . . . . . 53
3.6 The diagonal of a third-order cubical tensor. . . . . . . . . . 53
3.7 Visual illustration of the mode-1 unfolding. . . . . . . . . . 54
3.8 Visual illustration of the mode-n (mode-1) multiplication. . 54
3.9 The CANDECOMP/PARAFAC decomposition of a third-
order tensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.10 The Tucker decomposition of a third-order tensor. . . . . . . 58

xiii
xiv List of Figures

3.11 Illustration of (a) vector-to-vector projection, (b) tensor-to-

tensor projection, and (c) tensor-to-vector projection, where
EMP stands for elementary multilinear projection. . . . . . 60
3.12 Illustration of an elementary multilinear projection. . . . . . 62
3.13 Comparison of the number of parameters to be estimated by
VVP, TVP, and TTP, normalized with respect to the number
by VVP for visualization. . . . . . . . . . . . . . . . . . . . 65

4.1 The multilinear subspace learning (MSL) framework. . . . . 71

4.2 A taxonomy of PCA-based MSL algorithms. . . . . . . . . . 74
4.3 A taxonomy of LDA-based MSL algorithms. . . . . . . . . . 77
4.4 Overview of the history of tensor decomposition and multilin-
ear subspace learning. . . . . . . . . . . . . . . . . . . . . . 78

5.1 Typical flow of a multilinear subspace learning algorithm. . 89

5.2 Visual interpretation of (a) the total scatter tensor, (b) the
mode-1 eigenvalues, (c) the mode-2 eigenvalues, and (d) the
mode-3 eigenvalues of the respective mode-n total scatter ma-
trix for input samples. . . . . . . . . . . . . . . . . . . . . . 95
5.3 Plots of (a) the eigenvalue magnitudes, and (b) their cumula-
tive distributions for synthetic datasets db1, db2, and db3. . 98
5.4 Plots of (a) the eigenvalue magnitudes and (b) their cumula-
tive distributions for the gallery set of the USF Gait database
V.1.7. (up to thirty eigenvalues). . . . . . . . . . . . . . . . 99

6.1 Multilinear PCA algorithms under the MSL framework. . . 107

6.2 Illustration of recognition through LDA-style boosting with
regularization on MPCA features. . . . . . . . . . . . . . . . 131

7.1 Multilinear discriminant analysis algorithms under the MSL

framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

8.1 Multilinear ICA, CCA, and PLS algorithms under the MSL
framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
8.2 The structured data in (a) are all mixtures generated from
the source data in (b) with a multilinear mixing model. . . . 167
8.3 Blind source separation by MMICA on synthetic data. . . . 172
8.4 Schematic of multilinear CCA for paired (second-order) tensor
datasets with two architectures. . . . . . . . . . . . . . . . . 178

9.1 A typical pattern recognition system. . . . . . . . . . . . . . 189

9.2 A face image represented as a second-order tensor (matrix) of
column × row. . . . . . . . . . . . . . . . . . . . . . . . . . 192
9.3 Face recognition results by supervised subspace learning algo-
rithms on the PIE database. . . . . . . . . . . . . . . . . . . 194
 List of Figures xv

9.4 Face recognition results by unsupervised subspace learning

algorithms on the FERET database. . . . . . . . . . . . . . 195
9.5 A gait silhouette sequence as a third-order tensor of column×
row × time. . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.6 The correct recognition rates of supervised subspace learning
algorithms on the 32 × 22 × 10 USF gait database V.1.7. . . 198
9.7 Multichannel electroencephalography (EEG) signals with
each channel as a time series recorded by an electrode placed
on the scalp. . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
9.8 A functional magnetic resonance imaging (fMRI) scan se-
quence with three spatial modes and one temporal mode . . 201
9.9 A 2D auditory spectrogram representation of music signals. 203
9.10 Network traffic data organized as a third-order tensor of
source IP×destination IP×port number. . . . . . . . . . . . 203

B.1 Illustration of face image preprocessing. . . . . . . . . . . . 221

B.2 Sample face images of one subject from the CMU PIE
database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
B.3 Examples of face images from two subjects in the FERET
database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
B.4 Sample frames from the Gait Challenge datasets. . . . . . . 223
B.5 Illustration of the silhouette extraction process. . . . . . . . 224
B.6 Three gait samples from the USF Gait database V.1.7, shown
by concatenating frames in rows. . . . . . . . . . . . . . . . 225
List of Tables

2.1 PCA, LDA, CCA and PLS can all be viewed as solving the
generalized eigenvalue problem Av = λBv. . . . . . . . . . 39

3.1 Number of parameters to be estimated by three multilinear

projections. . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.1 Linear versus multilinear subspace learning. . . . . . . . . . 73

5.1 Order of computational complexity of eigendecomposition

for multilinear subspace learning (MSL) and linear subspace
learning (LSL). . . . . . . . . . . . . . . . . . . . . . . . . . 102

9.1 Six distance (dissimilarity) measures d(a, b) between feature

vectors a ∈ RH and b ∈ RH , with an optional weight vector
w ∈ RH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

B.1 Characteristics of the gait data from the USF Gait Challenge
datasets version 1.7. . . . . . . . . . . . . . . . . . . . . . . 224

xvii
List of Algorithms

2.1 Principal component analysis (PCA) . . . . . . . . . . . . . . 23

2.2 Linear discriminant analysis (LDA) . . . . . . . . . . . . . . . 30
2.3 Canonical correlation analysis (CCA) . . . . . . . . . . . . . . 35
2.4 Nonlinear iterative partial least squares (NIPALS) . . . . . . 37
2.5 PLS1 regression . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.6 Adaptive boosting (AdaBoost) . . . . . . . . . . . . . . . . . 44
5.1 A typical TTP-based multilinear subspace learning algorithm 91
5.2 A typical TVP-based multilinear subspace learning algorithm 92
6.1 Generalized PCA (GPCA) . . . . . . . . . . . . . . . . . . . . 111
6.2 Multilinear principal component analysis (MPCA) . . . . . . 117
6.3 Tensor rank-one decomposition (TROD) . . . . . . . . . . . . 123
6.4 Uncorrelated multilinear PCA (UMPCA) . . . . . . . . . . . 127
6.5 LDA-style booster based on MPCA features . . . . . . . . . . 133
7.1 Two-dimensional LDA (2DLDA) . . . . . . . . . . . . . . . . 145
7.2 Discriminant analysis with tensor representation (DATER) . 148
7.3 General tensor discriminant analysis (GTDA) . . . . . . . . . 149
7.4 Tensor rank-one discriminant analysis (TR1DA) . . . . . . . 152
7.5 Regularized uncorrelated multilinear discriminant analysis (R-
UMLDA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.6 Regularized UMLDA with aggregation (R-UMLDA-A) . . . . 161
8.1 Multilinear modewise ICA (MMICA) . . . . . . . . . . . . . . 170
8.2 Two-dimensional CCA (2D-CCA) . . . . . . . . . . . . . . . . 175
8.3 Multilinear CCA for matrix sets (Architecture I) . . . . . . . 183
8.4 Tri-linear PLS1 [N -way PLS (N -PLS)] . . . . . . . . . . . . . 186
8.5 Higher-order PLS (HOPLS) . . . . . . . . . . . . . . . . . . . 187

xix
Acronyms and Symbols

Acronym Description
AdaBoost Adaptive boosting
ALS Alternating least squares
APP Alternating partial projections
BSS Blind source separation
CANDECOMP Canonical decomposition
CCA Canonical correlation analysis
CRR Correct recognition rate
DATER Discriminant analysis with tensor representation
EMP Elementary multilinear projection
FPT Full projection truncation
GPCA Generalized PCA
GTDA General tensor discriminant analysis
HOPLS Higher-order PLS
HOSVD High-order SVD
IC Independent component
ICA Independent component analysis
LDA Linear discriminant analysis
LSL Linear subspace learning
MCCA Multilinear CCA
MMICA Multilinear modewise ICA
MPCA Multilinear PCA
MSL Multilinear subspace learning
NIPALS Nonlinear iterative partial least squares
NMF Nonnegative matrix factorization
N -PLS N -way PLS
NTF Nonnegative tensor factorization
PARAFAC Parallel factors
PC Principal component
PCA Principle component analysis
PLS Partial least squares
R-UMLDA Regularized UMLDA
R-UMLDA-A Regularized UMLDA with aggregation
SMT Sequential mode truncation
SSS Small sample size
SVD Singular value decomposition
SVM Support vector machine
TR1DA Tensor rank-one discriminant analysis
TROD Tensor rank-one decomposition

xxi
xxii Acronyms and Symbols

Acronym Description
TTP Tensor-to-tensor projection
TVP Tensor-to-vector projection
UMLDA Uncorrelated multilinear discriminant analysis
UMPCA Uncorrelated MPCA
VVP Vector-to-vector projection

Symbol Description
|A| Determinant of matrix A Jn Mode-n dimension for the
 · F Frobenius norm second set in CCA/PLS ex-
a or A A scalar tensions
a A vector K Maximum number of itera-
A A matrix tions
A A tensor k Iteration step index
Ā or Ā The mean of samples {Am } L Number of training samples
or {Am } for each class
AT Transpose of matrix A M Number of training samples
A−1 Inverse of matrix A m Index of training sample
A(i1 , i2 ) Entry at the i1 th row and Mc Number of training samples
i2 th column of A in class c
A(n) Mode-n unfolding of tensor
N Order of a tensor, number of
A
indices/modes
< A, B >Scalar product of A and B
n Mode index of a tensor
A ×n U Mode-n product of A by U
P Dimension of the output
a◦b Outer (tensor) product of a
vector, also number of EMPs
and b
in a TVP, or number of la-
C Number of classes
tent factors in PLS
c Class index
cm Class label for the mth Pn Mode-n dimension in the
training sample, the mth el- projected (output) space of
ement of the class vector c a TTP
δpq Kronecker delta, δpq = 1 iff p Index of the output vector,
p = q and 0 otherwise also index of the EMP in a
∂f (x)
Partial derivative of f with TVP, or index of latent fac-
∂x
respect to x tor in PLS
gp The pth coordinate vector ΨB Between-class scatter (mea-
g pm gp (m), the mth element of sure)
gp , see ymp ΨT Total scatter (measure)
Hy Number of selected features ΨW Within-class scatter (mea-
in MSL sure)
I An identity matrix Q Ratio of total scatter kept in
In Mode-n dimension or mode- each mode
n dimension for the first set R The set of real numbers
in CCA/PLS extensions rm The (TVP) projection of
 Acronyms and Symbols xxiii

the first set sample Xm in or the pth mode-1 projec-

second-order MCCA tion vector for the first set in
ρ Sample Pearson correlation second-order MCCA, or the
SB Between-class scatter matrix pth latent vector in HOPLS
in LSL u yp The pth projection vec-
(n)
SB Mode-n between-class scat- tor for the second set in
ter matrix in MSL CCA/PLS, or the pth mode-
SByp Between-class scatter of pth 1 projection vector for the
EMP projections {ymp , m = second set in second-order
1, ..., M } MCCA
(n)
ST Total scatter matrix in LSL {up } The pth EMP in a TVP, con-
(n)
ST Mode-n total scatter matrix sisting of N projection vec-
in MSL tors
(n)
STyp Total scatter of pth EMP {up }P
N A TVP, consisting of P
projections {ymp , m = EMPs (P × N projection
1, ..., M } vectors)
SW Within-class scatter matrix vec(A) Vectorized representation of
in LSL a tensor A
(n)
SW Mode-n within-class scatter v xp The pth mode-2 projection
matrix in MSL vector for the first set in
SWyp Within-class scatter of second-order MCCA
the pth EMP projections v yp The pth mode-2 projection
{ymp , m = 1, ..., M } vector for the second set in
sm The (TVP) projection of the second-order MCCA
second set sample Ym in wp The pth coordinate vector
second-order MCCA for the first set in CCA/PLS
tr(A) The trace of matrix A or second-order MCCA, or
Xm The mth input tensor sam- the pth latent factor in tri-
ple linear PLS1 (N -PLS)
xm The mth input vector sam- Xm The mth (training/input)
ple or the mth sample in the tensor sample
first set in CCA/PLS Ym Projection of Xm on a TTP
U Projection matrix in LSL {U(n) }, or the mth sam-
Ũ or ũ The (sub)optimal solution of ple in the second set in
U or u CCA/PLS extensions
U(n) Mode-n projection matrix Ý (n) Mode-n partial multilinear
{U(n) } A TTP, consisting of N pro- projection of raw samples in
jection matrices TTP
u(n) Mode-n projection vector Ŷ (n) Mode-n partial multilinear
up The pth projection vector in projection of centered (zero-
LSL, or the pth mode-2 pro- mean) samples in TTP
jection vector in tri-linear ym Vector projection of Xm (re-
PLS1 (N -PLS) arranged from TTP projec-
u xp The pth projection vector tion Ym in TTP-based MSL
for the first set in CCA/PLS, or projection on a TVP in
xxiv Acronyms and Symbols

TVP-based MSL), or the mean) samples in the pth

mth sample in the second set EMP of a TVP
in CCA/PLS ymp = ym (p) = gp (m), projec-
(n) tion of Xm on the pth EMP
ýp Mode-n partial multilinear (n)
{up }
projection of raw samples in
zp The pth coordinate vec-
the pth EMP of a TVP
tor for the second set in
(n)
ŷp Mode-n partial multilinear CCA/PLS or second-order
projection of centered (zero- MCCA
Preface

With the advances in sensor, storage, and networking technologies, bigger

and bigger data are being generated on a daily basis in a wide range of ap-
plications, especially in emerging cloud computing, mobile Internet, and big
data applications. Most real-world data, either big or small, have multidimen-
sional representations. Two-dimensional (2D) data include gray-level images
in computer vision and image processing, multichannel electroencephalogra-
phy (EEG) signals in neuroscience and biomedical engineering, and gene ex-
pression data in bioinformatics. Three-dimensional (3D) data include 3D ob-
jects in generic object recognition, hyperspectral cube in remote sensing, and
gray-level video sequences in activity or gesture recognition for surveillance
and human–computer interaction. A functional magnetic resonance imaging
(fMRI) sequence in neuroimaging is an example of four-dimensional (4D) data.
Other multidimensional data appear in medical image analysis, content-based
retrieval, and space-time super-resolution. In addition, many streaming data
and mining data are frequently organized in multidimensional representations,
such as those in social network analysis, Web data mining, sensor network
analysis, and network forensics. Moreover, multiple features (e.g., different
image cues) can also form higher-order tensors in feature fusion.
These multidimensional data are usually very high-dimensional, with a
large amount of redundancy and occupying only a small subspace of the en-
tire input space. Therefore, dimensionality reduction is frequently employed
to map high-dimensional data to a low-dimensional space while retaining as
much information as possible. Linear subspace learning (LSL) algorithms are
traditional dimensionality reduction techniques that represent input data as
vectors and solve for an optimal linear mapping to a lower-dimensional space.
However, they often become inadequate when dealing with big multidimen-
sional data. They result in very high-dimensional vectors, lead to the estima-
tion of a large number of parameters, and also break the natural structure
and correlation in the original data.
Due to the above challenges, especially in emerging big data applications,
there has been an urgent need for more efficient dimensionality reduction
schemes for big multidimensional data. Consequently, there has been a growing
interest in multilinear subspace learning (MSL) that reduces the dimension-
ality of big data directly from their natural multidimensional representation:
tensors, which refer to multidimensional arrays here. The research on MSL
has progressed from heuristic exploration to systematic investigation, while

xxv
xxvi Preface

recent prevalence of big data applications has increased the demand for tech-
nical developments in this emerging research field. Thus, we found that there
is a strong need for a new book devoted to the fundamentals and foundations
of MSL, as well as MSL algorithms and their applications.
The primary goal of this book is to give a comprehensive introduction to
both theoretical and practical aspects of MSL for dimensionality reduction of
multidimensional data. It expects not only to detail recent advances in MSL,
but also to trace the history and explore future developments and emerging
applications. In particular, the emphasis is on the fundamental concepts and
system-level perspectives. This book provides a foundation upon which we can
build solutions for many of today’s most interesting and challenging problems
in big multidimensional data processing. Specifically, it includes the follow-
ing important topics in MSL: multilinear algebra fundamentals, multilinear
projections, MSL framework formulation, MSL optimality criterion construc-
tion, and MSL algorithms, solutions, and applications. The MSL framework
enables us to develop MSL algorithms systematically with various optimality
criteria. Under this unifying MSL framework, a number of MSL algorithms
are discussed and analyzed in detail. This book covers their applications in
various fields, and provides their pseudocodes and implementation tips to help
practitioners in further development, evaluation, and application. MATLAB R

source codes are made available online.

The topics covered in this book are of great relevance and importance
to both theoreticians and practitioners who are interested in learning com-
pact features from big multidimensional data in machine learning and pattern
recognition. Most examples given in this book highlight our own experiences,
which are directly relevant for researchers who work on applications in video
surveillance, biometrics, and object recognition. This book can be a useful
reference for researchers dealing with big multidimensional data in areas such
as computer vision, image processing, audio and speech processing, machine
learning, pattern recognition, data mining, remote sensing, neurotechnology,
bioinformatics, and biomedical engineering. It can also serve as a valuable re-
source for advanced courses in these areas. In addition, this book can serve
as a good reference for graduate students and instructors in the departments
of electrical engineering, computer engineering, computer science, biomedical
engineering, and bioinformatics whose orientation is in subjects where dimen-
sionality reduction of big multidimensional data is essential.
We organize this book into two parts. The “ingredients” are in Part I while
the “dishes” are in Part II. On the first page of each chapter, we include a
figure serving as a “graphic abstract” for the chapter wherever possible.
In summary, this book provides a foundation for solving many dimension-
ality reduction problems in multidimensional data applications. It is our hope
that its publication will foster more principled and successful applications of
MSL in a wide range of research disciplines.
We have set up the following websites for this book:
http://www.comp.hkbu.edu.hk/~haiping/MSL.html
 Preface xxvii

or
http://www.dsp.toronto.edu/~haiping/MSL.html
or
https://sites.google.com/site/tensormsl/
We will update these websites with open source software, possible corrections,
and any other useful materials to distribute after publication of this book.
The authors would like to thank the Edward S. Rogers Sr. Department
of Electrical and Computer Engineering, University of Toronto, for support-
ing this research work. H. Lu would like to thank the Institute for Infocomm
Research, the Agency for Science, Technology and Research (A*STAR), in
particular, How-Lung Eng, Cuntai Guan, Joo-Hwee Lim, and Yiqun Li, for
hosting him for almost four years. H. Lu would also like to thank the De-
partment of Computer Science, Hong Kong Baptist University, in particular,
Pong C. Yuen, and Jiming Liu for supporting this work. We thank Dimitrios
Hatzinakos, Raymond H. Kwong, and Emil M. Petriu for their help in our
work on this topic. We thank Kar-Ann Toh, Constantine Kotropoulos, An-
drew Teoh, and Althea Liang for reading through the draft and offering useful
comments and suggestions. We would also like to thank the many anonymous
reviewers of our papers who have given us tremendous help in advancing this
field. This book would not have been possible without the contributions from
other researchers in this field. In particular, we want to thank the following
researchers whose works have been particularly inspiring and helpful to us:
Lieven De Lathauwer, Tamara G. Kolda, Amnon Shashua, Jian Yang, Jieping
Ye, Xiaofei He, Deng Cai, Dacheng Tao, Shuicheng Yan, Dong Xu, and Xue-
long Li. We also thank editor Randi Cohen and the staff at CRC Press, Taylor
& Francis Group, for their support during the writing of this book.
Haiping Lu
Hong Kong

Konstantinos N. Plataniotis
Anastasios N. Venetsanopoulos
Toronto

For MATLAB
R product information, please contact:

The MathWorks, Inc.

3 Apple Hill Drive
Natick, MA 01760-2098 USA
Tel: 508-647-7000
Fax: 508-647-7001
E-mail: info@mathworks.com
Web: www.mathworks.com
Chapter 1
Introduction

With the advances in sensor, storage, and networking technologies, bigger and
bigger data are being generated daily in a wide range of applications. Figures
1.1 through 1.4 show some examples in computer vision, audio processing,
neuroscience, remote sensing, and data mining. To succeed in this era of big
data [Howe et al., 2008], it becomes more and more important to learn compact
features for efficient processing. Most big data are multidimensional and they
can often be represented as multidimensional arrays, which are referred to
as tensors in mathematics [Kolda and Bader, 2009]. Thus, tensor-based com-
putation is emerging, especially with the growth of mobile Internet [Lenhart
et al., 2010], cloud computing [Armbrust et al., 2010], and big data such as
the MapReduce model [Dean and Ghemawat, 2008; Kang et al., 2012].
This book deals with tensor-based learning of compact features from mul-
tidimensional data. In particular, we focus on multilinear subspace learning
(MSL) [Lu et al., 2011], a dimensionality reduction [Burges, 2010] method de-
veloped for tensor data. The objective of MSL is to learn a direct mapping from
high-dimensional tensor representations to low-dimensional vector/tensor rep-
resentations.

(a) (b) (c)

FIGURE 1.1: Examples of second-order tensor (matrix) data: (a)

a gray-level image, (b) multichannel electroencephalography (EEG)
signals (“Electroencephalography,” Wikipedia, the free encyclopedia,
http://en.wikipedia.org/wiki/Electroencephalography), (c) an auditory spec-
trogram.

1
2 Multilinear Subspace Learning

1.1 Tensor Representation of Multidimensional Data

Multidimensional data can be naturally represented as multidimensional
(multiway) arrays, which are referred to as tensors in mathematics [Lang,
1984; Kolda and Bader, 2009]. The number of dimensions (ways) N defines
the order of a tensor, and the elements (entries) of a tensor are addressed by
N indices. Each index defines one mode. Tensor is a generalization of vector
and matrix. Scalars are zero-order tensors, vectors are first-order tensors, ma-
trices are second-order tensors, and tensors of order three or higher (N ≥ 3)
are called higher-order tensors [De Lathauwer et al., 2000a; Kolda and Bader,
2009].

Tensor terminology: The term tensor has different meanings in math-

ematics and physics. The usage in this book refers to its meaning in math-
ematics, in particular multilinear algebra [De Lathauwer et al., 2000b,a;
Greub, 1967; Lang, 1984]. In physics, the same term generally refers to a
tensor field [Lebedev and Cloud, 2003], a generalization of a vector field.
It is an association of a tensor with each point of a geometric space and it
varies continuously with position.

Second-order tensor (matrix) data are two-dimensional (2D), with some

examples shown in Figure 1.1. Figure 1.1(a) shows a gray-level face image
in computer vision applications, with spatial column and row modes. Figure
1.1(b) depicts multichannel electroencephalography (EEG) signals in neuro-
science, where the two modes consist of channel and time. Figure 1.1(c) shows
an audio spectrogram in audio and speech processing with frequency and time
modes.
Third-order tensor data are three-dimensional (3D), with some examples
shown in Figure 1.2. Figure 1.2(a) is a 3D face object in computer vision
or computer graphics [Sahambi and Khorasani, 2003], with three modes of
(spatial) column, (spatial) row, and depth. Figure 1.2(b) shows a hyperspectral
cube in remote sensing [Renard and Bourennane, 2009], with three modes
of column, row, and spectral wavelength. Figure 1.2(c) depicts a binary gait
video sequence for activity or gesture recognition in computer vision or human-
computer interaction (HCI) [Chellappa et al., 2005; Green and Guan, 2004],
with the column, row, and time modes. Figure 1.2(d) illustrates social network
analysis data organized in three modes of conference, author, and keyword
[Sun et al., 2006]. Figures 1.2(e) and 1.2(f) demonstrate web graph mining data
organized in three modes of source, destination, and text, and environmental
sensor monitoring data organized in three modes of type, location, and time
[Faloutsos et al., 2007].
Similarly, fourth-order tensor data are four-dimensional (4D). Figure 1.3(a)
 Introduction 3

depicts a functional magnetic resonance imaging (fMRI) scan sequence in

brain mapping research [van de Ven et al., 2004]. It is a 4D object with four
modes: three spatial modes (column, row, and depth) and one temporal mode.
Another fourth-order tensor example is network traffic data with four modes:
source IP, destination IP, port number, and time [Kolda and Sun, 2008], as
illustrated in Figure 1.3(b).
Our tour through tensor data examples is not meant to be exhaustive.
Many other interesting tensor data have appeared and are emerging in a broad
spectrum of application domains including computational biology, chemistry,
physics, quantum computing, climate modeling, and control engineering [NSF,
2009].
Tensor for feature fusion: Moreover, multiple features of an image (and

(a) (b) (c)

(d) (e) (f)

FIGURE 1.2: Examples of third-order tensor data: (a) a 3D face im-

age (Source: www.dirk.colbry.com by Dr. Dirk Colbry), (b) a hyper-
spectral cube (“Hyperspectral imaging,” Wikipedia, the free encyclopedia,
http://en.wikipedia.org/wiki/Hyperspectral imaging), (c) a video sequence,
(d) social networks organized in conference×author×keyword, (e) web graphs
organized in source×destination×text, (f) environmental sensor monitoring
data organized in type×location×time.
4 Multilinear Subspace Learning

other data as well) can be represented as a third-order tensor where the first
two modes are column and row, and the third mode indexes different features
such that tensor is used as a feature combination/fusion scheme. For example,
local descriptors such as the Scale-Invariant Feature Transform (SIFT) [Lowe,
2004] and Histogram of Oriented Gradients (HOG) [Dalal and Triggs, 2005]
form a local descriptor tensor in [Han et al., 2012], which is shown to be more
efficient than the bag-of-feature (BOF) model [Sivic and Zisserman, 2003].
Local binary patterns [Ojala et al., 2002] on a Gaussian pyramid [Lindeberg,
1994] are employed to form feature tensors in [Ruiz-Hernandez et al., 2010a,b].
Gradient-based appearance cues are combined in a tensor form in [Wang et al.,
2011a], and wavelet transform [Antonini et al., 1992] and Gabor filters [Jain
and Farrokhnia, 1991] are used to generate higher-order tensors in [Li et al.,
2009a; Barnathan et al., 2010], and [Tao et al., 2007b], respectively. Figure 1.4

(a)

(b)

FIGURE 1.3: Examples of fourth-order tensor data: (a) a functional mag-

netic resonance imaging (fMRI) scan sequence with three spatial modes and
one temporal mode [Pantano et al., 2005], (b) network traffic data organized
in source IP×destination IP×port number×time.
 Introduction 5

FIGURE 1.4: A third-order tensor formed by the Gabor filter outputs of a

gray-level face image. Here, tensor is used as a feature fusion scheme.

shows an example of a third-order tensor formed by the Gabor filter outputs

of a gray-level face image.

1.2 Dimensionality Reduction via Subspace Learning

Real-world tensor data are commonly specified in a high-dimensional space.
Direct operation on this space suffers from the so-called curse of dimension-
ality:

• Handling high-dimensional data puts a high demand on processing

power and resources so it is computationally expensive [Shakhnarovich
and Moghaddam, 2004].

• When the number of data samples available is small compared to their

high dimensionality, that is, in the small sample size (SSS) scenario,
conventional tools become inadequate and many problems become ill-
posed or poorly conditioned [Ma et al., 2011].

Fortunately, these tensor data do not lie randomly in the high-dimensional

space; rather, they are highly constrained and confined to a subspace
[Shakhnarovich and Moghaddam, 2004; Zhang et al., 2004]. For example, as
shown in Figure 1.5, a 256-level facial image of 100 × 100 (on the left) is only
one of the 25610,000 points in the corresponding image space. As faces are
constrained with certain specific characteristics, all 256-level facial images of
100 × 100 will occupy only a very small portion, that is, a subspace, of the
corresponding image space. Thus, they are intrinsically low-dimensional and
there is lots of redundancy.
6 Multilinear Subspace Learning

FIGURE 1.5: A face can be represented as one point in an image space of

the same size. All 8-bit 100 × 100 faces occupy only a small portion, that is,
a subspace, of the whole image space of size 100 × 100 with 8 bits per pixel.

Dimensionality reduction 1 is an attempt to transform a high-dimensional

dataset into a low-dimensional representation while retaining most of the infor-
mation regarding the underlying structure or the actual physical phenomenon
[Law and Jain, 2006]. In other words, in dimensionality reduction, we are learn-
ing a mapping from high-dimensional input space to low-dimensional output
space that is a subspace of the input space, that is, we are doing subspace
learning. We can view the low-dimensional representation as latent variables
to estimate. Also, we can view this as a feature extraction process and the low-
dimensional representation as the features learned. These features can then
be used to perform various tasks, for example, they can be fed into a classifier
to identify its class label.

“In an information-rich world, the wealth of information

means a dearth of something else: a scarcity of what-
ever it is that information consumes. What information
consumes is rather obvious: it consumes the attention of
its recipients. Hence a wealth of information creates a
poverty of attention and a need to allocate that atten-
tion efficiently among the overabundance of information
sources that might consume it.”

Herbert Simon (1916–2001)

Economist, Turing Award Winner, and Nobel Laureate

Traditional subspace learning algorithms are linear ones operating on vec-

tors, that is, first-order tensors, including principal component analysis (PCA)
[Jolliffe, 2002], independent component analysis (ICA) [Hyvärinen et al.,
1 Dimensionality reduction is also known as dimension reduction or dimensional re-

duction [Burges, 2010]. Here, we adopt the name most commonly known in the machine
learning and pattern recognition literature.
 Introduction 7

(a) (b)

FIGURE 1.6: Linear versus multilinear mapping: (a) a tensor needs to be

reshaped before a linear mapping to a low-dimensional vector, (b) direct multi-
linear mapping from a tensor to a vector or another tensor of lower dimension.

FIGURE 1.7: Reshaping (vectorization) of a 32× 32 face image to a 1024× 1

vector breaks the natural structure and correlation in the original face image.
The 2D and 1D representations are shown on the same scale here.

2001], linear discriminant analysis (LDA) [Duda et al., 2001], canonical cor-
relation analysis (CCA) [Hotelling, 1936], and partial least squares (PLS)
analysis [Wold et al., 2001]. To apply these linear subspace learning (LSL)
methods on tensor data of order higher than one, such as images and videos,
we have to reshape (vectorize) tensors into vectors first, that is, to convert
N -dimensional arrays (N > 1) to one-dimensional arrays, as depicted in Fig-
ure 1.6(a). Thus, LSL only partly alleviates the curse of dimensionality while
such reshaping, (i.e., vectorization), has two fundamental limitations:
• Vectorization breaks the natural structure and correlation in the original
data, reduces redundancies and/or higher order dependencies present
in the original dataset, and loses potentially more compact or useful
representations that can be obtained in the original tensor forms. For
example, Figure 1.7 shows a 2D face image of size 32 × 32 and its cor-
responding reshaped vector with size 1024 × 1 on the same scale. From
the 2D image (matrix) representation above, we can tell it is a face.
However, from the 1D vector representation below, we cannot tell what
it is.
• For higher-order tensor data of large size such as video sequences, the
reshaped vectors are very high dimensional. Analysis of these vectors
8 Multilinear Subspace Learning

(a)

(b)

FIGURE 1.8: Vector-based versus tensor-based analysis of a 3D object:

(a) reshaping (vectorization) of a 128 × 88 × 20 gait silhouette sequence to
a 225, 280 × 1 vector will lead to a covariance (or scatter) matrix of size
225, 280 × 225, 280, which is about 189GB using floating-point data type; (b)
tensor-based processing of the same gait sequence will lead to three covariance
matrices of size 128 × 128, 88 × 88, and 20 × 20. The total size of these three
covariance matrices will be about 95.8KB using floating-point data type.

often leads to high (sometimes impractical) computational and memory

demand, and results in the SSS difficulties due to a large number of
parameters to be estimated. For example, Figure 1.8(a) shows a 3D
 Introduction 9

gait silhouette sequence of size 128 × 88 × 20 on the top. Its vectorized

version has a size of 225, 280 × 1. In statistical analysis, we often need
to calculate the covariance (or scatter) matrix, which will have a size of
225, 280 × 225, 280 for the vectorized gait sequence. If we use a floating-
point data type for the covariance matrix, it will need about 2.0 × 1011
bytes of memory, which is about 189GB. Although there are tricks
to avoid such a big covariance matrix when it is of low-rank [Turk and
Pentland, 1991], these tricks may have limited applicability or usefulness
as more and more data are available in an era of big data.

Therefore, there has been a surging interest in more effective and efficient
dimensionality reduction schemes for tensor data. Methods working directly
on tensor representations have emerged as a promising approach. When we
use tensor-based analysis, tensors are processed directly without vectoriza-
tion. Figure 1.8(b) shows tensor-based analysis of the same 3D object in Fig-
ure 1.8(a), leading to three covariance matrices of size 128 × 128, 88 × 88, and
20 × 20. The total size of these three covariance matrices will be only about
95.8KB, which is several orders of magnitude smaller than the size of the co-
variance matrix in vector-based processing (95.8KB/189GB≈ 4.8 × 10−7 ). In
other words, vector-based processing will need about 2 millions times more
memory for the covariance matrix than that of tensor-based processing in this
case.

1.3 Multilinear Mapping for Subspace Learning

Driven by a proliferation of data-intensive applications, researchers have been
working on broadening and generalizing successful statistical and computa-
tional tools based on linear algebra. Recently, the National Science Foundation
(NSF) is calling for advancement in “computational thinking.” In the report
of the 2009 NSF Workshop on “Future Directions in Tensor-Based Computa-
tion and Modeling,” it was pointed out that in matrix computations, the level
of thinking seems to “kick up” about every twenty years as shown in Figure
1.9 [NSF, 2009]. We have progressed from scalar-level thinking to matrix level
thinking, and then to block matrix -level thinking. Now we are in a transition
from matrix-based to tensor-based computational thinking.
From a machine learning perspective, the goal of this book is to advance
tensor-level computational thinking in dimensionality reduction via subspace
learning. We investigate multilinear subspace learning (MSL) of compact rep-
resentations for multidimensional data.
MSL solves for a multilinear mapping, which transforms input tensor data
directly to low-dimensional tensors of the same or lower order (e.g., vectors), as
10 Multilinear Subspace Learning

FIGURE 1.9: The field of matrix computations seems to “kick up” its level of
thinking about every 20 years. (Adapted from the report of the NSF Workshop
on “Future Directions in Tensor-Based Computation and Modeling,” 2009
[NSF, 2009].)

shown in Figure 1.6(b). In contrast with linear mapping, multilinear mapping

does not reshape tensors into vectors. Each basis in a linear mapping of tensor
data is often specified by a large number of parameters, which equals to the
input dimensionality, that is, product of dimensions in each mode. In contrast,
each basis in a multilinear mapping of the same tensor data is often specified by
a much smaller number of parameters, which equals to the sum of dimensions
in each mode. Thus, bases in linear mapping have large degrees of freedom
so they are able to capture richer representations, while bases in multilinear
mapping are more constrained so the learned representations are sparser with
more “grid-like” structures or regularities. This brings MSL three key benefits
[Lu et al., 2011]:

• It preserves data structure before mapping by taking tensors directly as

input.

• It can learn more compact and potentially more useful representations

than linear subspace learning. With the same amount of data, MSL has
no or a much less severe small sample size problem than LSL.

• It can handle big tensor data more efficiently with computations in much
lower dimensions than linear methods.

Next, we demonstrate the different characteristics of features learned

through linear and multilinear subspace learning with two examples. Figure
1.10 shows a second-order example of subspace learning on 1,360 face images.
The input face images are of size 80 × 80 = 6, 400, with a sample shown in
Figure 1.10(a). Figures 1.10(b) and 1.10(c) depict three most discriminative
face bases learned through linear and multilinear subspace learning using the
 Introduction 11

same principle, respectively. A linear basis requires 6,400 parameters to specify

while a multilinear basis needs only 160 parameters to specify, which is more
160 1
compact ( 6400 = 40 ). Consequently, the linear bases look like “ghost-faces”
while the multilinear bases have simpler “grid-like” structures. Despite being
simpler, the multilinear bases have been shown to have higher face recognition
accuracy in [Ye, 2005a].
Figure 1.11 is a third-order example on 731 gait silhouette sequences of
size 64 × 44 × 20 = 56, 320, with a sample shown in Figure 1.11(a). Figures
1.11(b) and 1.11(c) show three most discriminative gait bases learned through
linear and multilinear subspace learning using the same principle, respectively.
In this case, each linear basis requires 56,320 parameters to specify, while a
multilinear basis needs only 128 parameters to specify, which is more compact
128 1
( 56320 = 440 ). Similar to the second-order case in Figure 1.10, the linear bases
closely resemble a real gait sequence in Figure 1.11(a) while the multilinear
bases look more like filter banks with simpler “grid-like” structures. These
multilinear bases have been shown to give higher gait recognition accuracy in
[Lu, 2008; Lu et al., 2008b].

1.4 Roadmap
This book aims to provide a systematic and unified treatment of multilinear
subspace learning. As Hamming [1986] suggested in his inspiring talk “You
and Your Research,” we want to provide the essence of this specific field.
This first chapter has provided the motivation for MSL and a brief introduc-
tion to it. We started with the tensor representation of multidimensional data

(a) (b) (c)

FIGURE 1.10: Illustration of second-order feature characteristics: (a) a sam-

ple face image of size 80 × 80, (b) three most discriminative face bases learned
by linear subspace learning where each basis is specified by 6,400 parameters,
and (c) three most discriminative face bases learned by multilinear subspace
learning using a similar optimality criterion, where each basis is specified by
160 parameters. These simpler representations are shown to have higher face
recognition accuracy in [Ye, 2005a].
12 Multilinear Subspace Learning

and the need for dimensionality reduction to deal with the curse of dimen-
sionality. Then, we discussed how conventional linear subspace learning for
dimensionality reduction becomes inadequate for big tensor data as it needs
to reshape tensors into high-dimensional vectors. From tensor-level computa-
tional thinking, MSL was next introduced to learn compact representations
through direction multilinear mapping of tensors to alleviate those difficulties
encountered by their linear counterparts.
The rest of this book consists of two parts:
Part I covers the fundamentals and foundations of MSL. Chapter
2 reviews five basic LSL algorithms: PCA, ICA, LDA, CCA, and PLS. It

(a)

(b)

(c)

FIGURE 1.11: Illustration of third-order feature characteristics: (a) a sam-

ple gait silhouette sequence of size 64 × 44 × 20, (b) three most discriminative
gait bases (one in each row) learned by linear subspace learning where each ba-
sis is specified by 56,320 parameters, (c) three most discriminative gait bases
(one in each row) learned by multilinear subspace learning using a similar op-
timality criterion, where each basis is specified by 128 parameters. These more
compact representations are shown to give higher gait recognition accuracy in
[Lu et al., 2008b].
 Introduction 13

also discusses closely-related topics including regularization, model selection,

and ensemble-based learning. Chapter 3 presents multilinear algebra pre-
liminaries and two popular tensor decompositions in the beginning. Then, it
introduces multilinear projections and scatter measures commonly used for
constructing optimization criteria. These two chapters lay the groundwork
necessary for the MSL algorithms in Part II. Chapter 4 develops the MSL
framework and gives an overview of PCA-based and LDA-based MSL algo-
rithms. It also traces the history of MSL and related works, and points out
future research directions. Chapter 5 considers the algorithmic and compu-
tational aspects including typical solutions for MSL and related issues such
as initialization and termination, synthetic data generation, feature selection
strategies, resource demand, computational complexity, and some implemen-
tation tips for large datasets. It is an important chapter for practitioners who
wants to employ or further develop MSL algorithms in their applications of
interest.

“In this day of practically infinite knowledge, we need

orientation to find our way. ... And we cope with that,
essentially, by specialization. ... The present growth of
knowledge will choke itself off until we get different tools. I
believe that books which try to digest, coordinate, get rid
of the duplication, get rid of the less fruitful methods and
present the underlying ideas clearly of what we know now,
will be the things the future generations will value. ...
But I am inclined to believe that, in the long-haul, books
which leave out what’s not essential are more important
than books which tell you everything because you don’t
want to know everything. ... You just want to know the
essence.”

Richard Hamming (1915–1998)

Mathematician, Turing Award Winner

Part II presents specific MSL algorithms and applications. Chapters

6 and 7 are devoted to multilinear extensions of PCA and LDA, respectively.
Chapter 8 covers other MSL algorithms including multilinear extensions of
ICA, CCA, and PLS. Within each chapter, five selected algorithms will be
treated with more details for a deeper understanding of the subject. Lastly,
Chapter 9 describes a typical pattern recognition system and examines var-
ious applications of MSL. This chapter includes some experimental perfor-
14 Multilinear Subspace Learning

mance comparisons of MSL algorithms on popular face and gait recognition

problems.
Appendix A gives some mathematical background for reference. Ap-
pendix B provides details on popular face and gait databases for MSL and
the necessary preprocessing steps involved. Finally, Appendix C points the
readers to related open source software and discusses the motivations, benefits,
and common concerns of open source, as well as some software development
tips.
On the first page of each chapter, we try to provide a figure that can (hope-
fully) act as a graphical abstract2 to illustrate the most important concept in
the chapter. At the end of each chapter in Part I, we summarize the key points
and provide further readings. As Part II deals with specific algorithms and
applications, further readings will be the respective references for chapters in
this part. In addition, the contents of Chapter 2 are covered in many exist-
ing books on machine learning with varying levels of detail so knowledgeable
readers may skip this chapter on first read.
This book focuses on machine learning. However, machine learning algo-
rithms alone are often not enough to make a real impact in real-world appli-
cations, as pointed out in [Wagstaff, 2012]. To build a working system based
on machine learning, it is important to formulate the real-world problems into
machine learning tasks, collect data to learn from, and preprocess raw data
into a form suitable for learning. Furthermore, it is also important to evaluate
the learning system properly, give insightful interpretations, communicate the
results to relevant scientific community, and persuade potential users to adopt
the methods or systems.

1.5 Summary
• Most big data are multidimensional.

• Multidimensional data are called tensors, and each dimension is called

a mode.
• Dimensionality reduction is often needed to transform tensors into lower
dimension by learning a mapping to a subspace.

• Linear subspace learning needs to reshape input tensors into vectors,

requiring lots of parameters to be estimated. This leads to difficulties
when dealing with big tensor data.
2 This is inspired by the graphical abstracts introduced by Elsevier:
http://www.elsevier.com/authors/graphical-abstract
 Introduction 15

• Multilinear subspace learning directly maps tensor data to lower di-

mension. It can preserve data structure, result in more compact rep-
resentations, and lead to lower computational demand than its linear
counterpart.
 Part I

Fundamentals and
Foundations
Chapter 2
Linear Subspace Learning for
Dimensionality Reduction

In Chapter 1, we pointed out why dimensionality reduction is commonly

needed in practice. On one hand, real-world data are often specified in a
high-dimensional space. Direct processing of high-dimensional data is com-
putationally expensive and the number of samples available is often small
compared to their high dimensionality. On the other hand, real-world data
are often highly constrained to a low-dimensional subspace. Thus, dimension-
ality reduction attempts to learn a mapping of high-dimensional data to a
low-dimensional space, that is, a subspace.
This chapter introduces the basics of linear subspace learning (LSL) for
dimensionality reduction and serves as the foundation for Chapters 6, 7, and
8. LSL solves for a linear mapping to a subspace by optimizing a criterion
[Duda et al., 2001; Shakhnarovich and Moghaddam, 2004]. It takes a high-
dimensional random vector x as input and transforms it to a low-dimensional
vector y through a projection matrix U, as shown in Figure 2.1. When the
input data are multidimensional, they need to be reshaped into vectors first.
LSL captures some population properties of the random vector x [Jolliffe,
2002]. In practice, the mapping or projection is learned from only a sample (a
selected/collected subset) of the (whole) population [Anderson, 2003], e.g., a
dataset {x1 , x2 , x3 , ...}. Throughout this book, learning is with respect to the
sample rather than the population.

FIGURE 2.1: Linear subspace learning solves for a projection matrix U,

which maps a high-dimensional vector x to a low-dimensional vector y.

19
20 Multilinear Subspace Learning

We first consider five basic LSL algorithms, namely principal compo-

nent analysis (PCA), independent component analysis (ICA), linear discrim-
inant analysis (LDA), canonical correlation analysis (CCA), and partial least
squares (PLS) analysis, in Sections. 2.1 to 2.5. PCA and ICA are unsupervised
learning algorithms, which take unlabeled data as the input. LDA is a super-
vised learning algorithm that requires labeled data as input. While PCA, ICA,
and LDA learn a subspace for one dataset, CCA and PLS learn two subspaces
for two (paired) datasets. CCA and PLS are unsupervised learning algorithms
by design. However, they are often used as supervised learning algorithms
where one set of data is simply the labels.
After presenting these five algorithms, we give a unified view of PCA, LDA,
CCA, and PLS as generalized eigenvalue problems. Next, we discuss several
closely related topics, including regularization for reducing overfitting and
improving generalization, cross-validation for model selection, and ensemble
methods, where multiple models or learners are combined to obtain better
performance.

2.1 Principal Component Analysis

PCA reduces the dimensionality of an input dataset consisting of correlated
variables into an output set of linearly uncorrelated variables, the principal
components (PCs). The output set, with reduced dimensionality, retains as
much as possible the variation (or energy) present in the original input set1 .
The PCs are usually ordered by the variation captured in descending order
and the tail values can be truncated for a desirable amount of dimension-
ality reduction. PCs are independent only if the dataset is jointly normally
distributed [Jolliffe, 2002]. PCA is an unsupervised learning method. In the
following, we show a standard derivation of PCA.
Suppose that the input dataset has M samples {x1 , x2 , ..., xM }, where each
sample xm ∈ RI is a vector, m = 1, ..., M . We measure the variation through
the total scatter matrix ST simply as (m − 1) times the sample covariance
matrix [Duda et al., 2001],


M
ST = (xm − x̄)(xm − x̄)T , (2.1)
m=1

1 One of the rationales behind capturing the maximal variation is that noise can be

assumed to be uniformly spread so directions of high variation tend to have a higher signal-
to-noise ratio [Bie et al., 2005].
 Linear Subspace Learning for Dimensionality Reduction 21

where x̄ is the sample mean defined as

1 
M
x̄ = xm . (2.2)
M m=1

In PCA, we want to find a linear mapping of the centered samples {xm − x̄}
to a low-dimensional representation {ym ∈ RP }, P < I, through a projection
matrix U ∈ RI×P as
ym = UT (xm − x̄). (2.3)
The objective of PCA is to maximize the variation captured by the projected
samples (or extracted features) {ym }. The projection matrix U can be consid-
ered to consist of P projection directions {u1 , u2 , ..., uP }. Then the projection
in each direction is
ymp = uTp (xm − x̄), (2.4)
where p = 1, ..., P . Let us define a coordinate vector gp ∈ RM for each p,
where the mth element of gp is defined as

gpm = uTp (xm − x̄) = ymp . (2.5)

PCA requires that each direction maximizes uTp ST up , and gp and gq are
uncorrelated for q = p, p, q = 1, ..., P .

Centering: The subtraction of the mean in Equations (2.1) and (2.3) is

called centering. It is often carried out as a preprocessing procedure. When
the mean is zero (x̄ = 0), the total scatter matrix becomes


M
ST = xm xTm = XXT , (2.6)
m=1

where X ∈ RI×M is a data matrix formed with the M input samples

{xm } as its columns. Sometimes, Equation (2.6) is used directly without
centering even when the mean is nonzero (x̄ = 0). It should be noted that
centering is preferred in PCA to ensure that the first PC describes the
direction of maximum variation. If centering is not performed, the first PC
often represents the mean of the data and the variation captured by the
first PC often accounts for over 99% of the total variation, as pointed out
in Section 3.4.3 of [Jackson, 1991]. Moreover, the further from the origin is
the mean, the larger will be the variation captured by the first PC relative
to that captured by the other PCs.

The first projection: To derive PCA, consider p = 1 to solve for u1

by maximizing uT1 ST u1 first. We need to impose a normalization constraint
22 Multilinear Subspace Learning

uT1 u1 = 1 since the maximum will not be achieved for finite u1 . The objective
then becomes
ũ1 = arg max uT1 ST u1 subject to uT1 u1 = 1. (2.7)
u1

Using the technique of Lagrange multipliers (Section A.3), we need to maxi-

mize
ψ1 = uT1 ST u1 − λ(uT1 u1 − 1), (2.8)
where λ is a Lagrange multiplier. We differentiate Equation (2.8) with respect
to u1 and set the result to zero as
∂ψ1
= ST u1 − λu1 = (ST − λI)u1 = 0, (2.9)
∂u1
where I is an identity matrix2 . Therefore, λ and u1 are an eigenvalue and its
corresponding eigenvector of ST , respectively (Section A.1.8). In addition, the
quantity to be maximized is
uT1 ST u1 = uT1 λu1 = λuT1 u1 = λ. (2.10)
Thus, we obtain ũ1 as the eigenvector associated with the largest eigenvalue
λ1 of ST .
The second projection: Next, we solve for the second projection vector
u2 that maximizes uT2 ST u2 subject to the constraint that the projections by
u2 (i.e., g2 ) are uncorrelated with those by u1 (i.e., g1 ). This zero-correlation
constraint can be written as
uT1 ST u2 = uT2 ST u1 = uT2 λ1 u1 = λ1 uT2 u1 = 0 ⇒ uT2 u1 = 0. (2.11)
Similar to the derivation of u1 , we maximize the following quantity by incor-
porating the normalization and zero-correlation constraints:
ψ2 = uT2 ST u2 − λ(uT2 u2 − 1) − μuT2 u1 , (2.12)
where λ and μ are Lagrange multipliers. We differentiate Equation (2.12) with
respect to u2 and set the result to zero as
∂ψ2
= ST u2 − λu2 − μu1 = 0. (2.13)
∂u2
Multiplication of Equation (2.13) on the left by uT1 gives
uT1 ST u2 − λuT1 u2 − μuT1 u1 = 0 ⇒ μ = 0, (2.14)
where the first two terms are zero by Equation (2.11) and uT1 u1 = 1. Again,
from Equations (2.13) and (2.14) we have
ST u2 − λu2 = (ST − λI)u2 = 0, (2.15)
2 As the dimension of the identity matrix should be clear from the context, we do not

indicate its dimension using subscript for simpler notation.

 Linear Subspace Learning for Dimensionality Reduction 23

Algorithm 2.1 Principal component analysis (PCA)

Input: A set of M training samples {x1 , x2 , ..., xM }, and the number of PCs
to estimate P or percentage of captured variation η to determine P by
Equation (2.17).
Process:
1: Calculate the mean x̄ from Equation (2.2).
2: Calculate the total scatter matrix ST from Equation (2.1).
3: Get the P eigenvectors associated with the P largest eigenvalues of ST to
form Ũ.
Output: The projection matrix Ũ.

and λ and u2 are an eigenvalue and its corresponding eigenvector of ST ,

respectively. Following Equation (2.10), we are maximizing uT2 ST u2 = λ.
Assuming that ST does not have repeated eigenvalues3 , λ should not equal to
λ1 due to the constraint uT2 u1 = 0. Therefore, we obtain ũ2 as the eigenvector
associated with the second largest eigenvalue λ2 of ST .
Following a similar derivation, we can get the other projection vectors ũp
(p = 3, ..., P ) as the pth eigenvector associated with the pth largest eigenvalue
λp of ST . Thus, the projection matrix Ũ is composed of the eigenvectors
corresponding to the P (P < I) largest eigenvalues of ST and the variation
captured by the pth PC is the pth eigenvalue λp . The projected PCA features
have the following properties [Fukunaga, 1990]:
1. The effectiveness of each feature, in terms of representing the input, is
determined by its corresponding eigenvalue.
2. The projected features are uncorrelated. If the input data is Gaussian
(i.e., normally distributed), the projected features are mutually indepen-
dent (i.e., their mutual information is maximized).
3. The P eigenvectors minimize the mean squared error in reconstruction
over all choices of P orthonormal basis vectors.
The projection of a test sample x in the PCA subspace is obtained as

y = ŨT (x − x̄). (2.16)

The number of PCs, P , is commonly determined by specifying the total vari-

ation (energy) to be captured. For example, suppose ST has β (non-zero)
eigenvalues {λ1 , ..., λβ }in total, the number of PCs to be kept to capture at
least η% of the total variation (energy) is
P
∗ p=1 λp η
P = arg min β > . (2.17)
P
p=1 λp
100

3 Please refer to Section 2.4 of [Jolliffe, 2002] for the case of repeated eigenvalues.
Random documents with unrelated
content Scribd suggests to you:
in the most common cases of the labor-pains, more able to advise the
sick person to innocent remedies, where there is no complication in
the disorder, than those half-bred or ignorant pretenders: but if
there is a complication, then there must absolutely be a good
physician called in, the expence of which should not be regretted,
since life is at stake.

N o w in such cases, a midwife, though never so skilful, will neither

be ashamed nor backward to require such aid: whereas a man-
midwife, the more ignorant he is, will be but the more careful of
concealing that ignorance, and from the most false prejudice that
both the faculties of physic and surgery are implicit ingraftments on
the profession of midwifery in a man, will rather let mother and child
perish, than call in that assistance, of which he will be ashamed to
confess his standing in any need. He will then rashly do the best he
can for his patient: but what will that best most probably be? Torture
and death; and that with perfect impunity. I say most probably, for
not even the most credulous, or the most zealous for the
appropriation of this profession to the male-sex, can hardly carry the
blindness of credulity and obstinacy the length of assenting in
earnest, that in the common run of men-practitioners you are to find
at once the man-midwife, the physician, and the surgeon. Whereas
women, fully sufficient for all cases but the very extraordinary ones
indeed, are ever ready to call for proper help, on the first alarm of
danger, of which too their apprehension is much more quick and just
than that of the men.
 O b j e c t i o n the Tenth.
T h e ignorance of the women is the cause of the little confidence
there is reposed in them.

ANSWER.

I f this objection was fairly stated, it should be said, that the

ignorance of the women in the art of destroying mother and child,
occasions their not being trusted so much as they deserve with the
office of saving both. In that art indeed of perpetrating double
murder with perfect impunity, under the sanction of the public
credulity, imposed upon by a vain parade of learning, I readily
confess the men superior to the women. I do more than confess it, I
will prove it; and how? even from their own writings and confession,
not extorted from them by the spirit of candor, but from an
interested desire of decrying or supplanting one another, in order to
self-recommendation.

I n fact, whoever will, with a competent degree of knowledge of the

subject, and of due impartiality, peruse the practical treatises of
midwifery, written by the most celebrated practitioners, some of
whom have so vainly pretended to the triple union of the characters
of man-midwife, surgeon and physician in one person, and it will be
found, that all their boasted superiority of erudition, has only led
them into the greater errors of practice, and the most barbarous
violences to nature.

B u t perhaps I exaggerate. Let the reader judge for himself, and

pronounce as his own reason shall dictate to him. Let him if he can
read without shuddering, the following quotation from one of the
most celebrated men-midwives of the age, Levret, p. 199. “Mauriceau
had invented a new tire-tête, which was to be introduced into that
part (the uterus). Peu or Pugh, like many others, made use of
different hooks (crochets) and La Motte opening the head with
scissors, scooped out the brain, &c. We read, with horror, in all these
authors, that they have extracted children, who, tho’ much maimed
or mutilated, have yet lived several hours.”

U p o n this many reflections will naturally occur. These children

thus destroyed, owed most probably their death neither to nature,
nor to the difficulties of the passage through which the launch is
made into our world, but to the labor being prematurely forced, and
the delivery effectuated by those torturous instruments, which at
once kill the child, and not seldom irreparably wound the mother in
the tender contexture of these parts. A midwife, with less learning
and more patience than those gentlemen, and well acquainted with
the power and custom of Nature to operate in some subjects,
sometimes more slowly, and in all ever more safely and gently than
art, would have left to nature, not without her tenderest assistance of
that nature, the expulsion of the child. A proper predisposal of the
passage, and direction of the posture, with an unremitting attention
to employ the fingers, so as not to lapse the critical moment of
operation, often never to be recovered with safety to mother and
child, would have, I repeat it, and appeal to common sense for the
probability thereof, saved the lives of those innocents, which thus fell
the victims of those learned experiments, with instruments, which,
by the way, be it remarked, none are so forward to use, as those who
are the loudest in exclaiming against the employ of them. And reason
good, if they exclaim against them, it is evidently in order to cover
their practice with them, against which the minds of their patients
must so naturally be revolted. But that exclaiming does not evidently
hinder their being used, when, the truth is, that if due care was
previously taken with the patients, those execrable substitutes to the
fingers need never be used at all.

B u t if these instrumentarians were called to account for their so

justly presumable massacres, what would be their defence? Most
certainly not the truth. One would not own, that in order to attend a
richer patient, or perhaps to return to his bottle, he had recourse to
his fatal instruments, to make the quicker riddance or effectual
dispatch; another would not confess, that he employed them purely
because his fund of patience was exhausted; some would not care to
allow, that they used them purely on the scheme of trying
experiments; and none of them would, you may be sure, plead guilty
of ignorance of better and more salutary methods. No! their wilful
error, or that want of skill, they would be sure to conceal under the
cloud of hard words and scientific jargon, in which they would dress
up their respective cases, and insult the ignorance of those silly good
women, who know no better than to deliver those of their own sex
with the help of their fingers and hands, and who are so undextrous,
as to have no notion of putting them to such unnecessary tortures
and risks, as are inseparable from the use of those iron and steel
instruments. Instruments which rarely fail of destroying the child, or
at least cruelly wounding it, and never but injure the mother, not
only in those exquisitely tender-textured parts, where they are so
blindly and ungovernably introduced; but in the often irrecoverable
dilatations of the external orifice, the vagina, and especially the
fourchette or frænum labiorum, all which, in general, they
considerably damage: and always originally without necessity. For if
through carelessness, if through an impatience, so much more
natural to men than to women, in a case and position of this nature;
if through ignorance of the critical minute of extraction, the occasion
of operating with the fingers has not been lapsed, any recourse to
instruments is perfectly unnecessary, and they will hardly ever
succeed where the subject is inaccessible to the fingers, without
having the worst of consequences to dread from them both to mother
and child. Nothing then can be worse for a man-midwife, than to be
tempted to any negligence, to any precipitation, to any ostentation,
in short, of expedition or of superiority of skill to that of the women,
by his having those instruments at hands, the doing without which is
at once so much better and safer, even by the confession of those
who use them nevertheless.

H o w greatly then is the ignorance of the midwives preferable to

such an use, as the male-practitioners commonly make of that deep
learning of theirs, which only misleads them, at the expence of
humanity! How over-compensated is that want of theoretical
knowledge, so unjustly reproached to women, since they profess a
sufficiency even of that knowledge; how over-compensated, I say, is
that supposed want, by that instinctive keenness of apprehension,
and ready dexterity of theirs in the manual operation, which in them
is a pure gift of nature, and to which not the utmost efforts of art or
experience can ever make the men arrive, for reasons which will be
made clearly appear in the two following considerations.

F i r s t, It will hardly be denied, that the art of midwifery requires

a regular training or education for it. The season of that education
can only be that of youth. And surely in that season precisely, the
very nature of the study excludes those of the male-sex, at the same
time, that there is nothing in it indecent or improper for the females
destined to that profession. This proposition will be more clearly
illustrated, by an appeal to the reader’s own sense and reason upon
what passes, and must necessarily pass in those hospitals for the
reception of lying-in women, where those of the male-sex are allowed
to attend for the sake of learning the profession.

T h i s Charity is indeed founded upon specious motives, but the

conduct of it would make humanity shudder, even where no violence
is expressly intended to humanity; and without the least forced or
uncharitable conclusion, may serve to demonstrate the impropriety
of attempting to throw the practical part of midwifery into the hands
of male-practitioners, the implicit consequence of which must be the
exclusion of the midwives, without any direct and formal exclusion of
them, but purely from the discouragement that will hinder any good
and able ones being formed in future. And that no thoroughgood
men-midwives, except perhaps two or three extraordinary men in a
whole nation, can ever be formed, the procedure at the lying-in
hospitals, open to men-pupils, such as it must of all necessity be
from the nature of the thing itself, without any the least reproach
herein meant to the worthy managers, will convince all who will
make an unprejudiced use of their judgment.

W e will then suppose a lying-in hospital, in which, for the sake of

training up men to the profession of midwives, there are young
pupils of the male-sex admitted to attend and learn the practical and
manual part of the business. To obtain this end, we will not say that
women of virtue and character are subjected to the inspection and
palpation of a set of youths, who perhaps pay largely for their
privilege of attendance; but we will grant, that the objects of this
charity are entirely women, who, though they may have
unfortunately forfeited their right to virtue, cannot however have lost
their claim to the protection of that humanity, which, besides the
great and most political attention due to population, pays especially a
tender regard to the innocent burthen, though of a guilty mother. Yet
among these wretched victims, there may be not a few who, if they
were not even to deserve more compassion than blame, for particular
circumstances of their ruin, in which the villainy of men has often a
much greater share than female frailty itself, cannot surely deserve
that all traces of modesty, or natural remains of regard for it, should
be utterly eradicated by that hard necessity of theirs to accept of a
charity, by which they must be abandoned up to the researches of a
set of young men, to whose approaches their age and sex must alone
give an air of petulance and wantonness not to be explained away, to
the satisfaction of the poor passive sufferer, by the goodness of the
intention. Every one must be sensible of the dreadful effects such a
treatment must have on the mind of a poor creature in that
condition, when the imagination is known to be the most weak, and
susceptible of the most dangerous impressions. At that critical time,
amidst all the terrors and apprehensions inseparable from her
situation, she is moreover exposed to the greatest indignity that can
be well imagined, that of serving for a pillar of manage to break
young men into the exercise of that most unmanly profession. Nay,
that very circumstance of the use she is put to, which she is in fact to
consider as a kind of valuable consideration by her paid for the relief
afforded her, and which in that light can scarce be called a charity;
that very circumstance, I say, of her submission, at all calls, and upon
all pretences of the pupils, being accounted for to her by the good
intention of it, will yet hardly pass on a wretched, frightened,
harrassed woman, who, whatever may be said to procure her tame
acquiescence, can scarcely, if she has a spark of female modesty left
in her, be reconciled to the grossness of such usage, whether she
considers herself as the butt of wantonness, or the victim of
experiments, or perhaps of both the one and the other. It is well if
she is defended by her ignorance from any idea of those dreadful
instruments, of the having practices tried upon her with which, her
circumstances might but too reasonably render her apprehensive,
since a needless resort to them may be too often presumed in the
course of practice, where the men are even paid for their assistence.
These the men-midwives may possibly indeed conceal from the sight
of their patients, but I defy him to conceal them from their wounded
imagination, if they are not wholly ignorant or can think at all.

Y e t in pure justice to all parties it should be observed, that,

besides many other points to be learned only by ocular inspection
and manual palpation, of which no theory by book or precepts can
convey satisfactory or adequate notions, that great and essential
point in our profession, a skill in what we call the Touching, is not to
be acquired without a frequent habit of recourse to the sexual parts
whence the indications are taken. And in this nothing but personal
experience can perfect the practitioner. But this admitted, only
proves the more clearly the utter impropriety of men addicting
themselves to this occupation. For, once more, most certainly the
season of acquiring the nicety of that faculty of Touching, besides
other requisites in the art, is for obvious reasons that of youth. Now
let any one figure to himself boys or young men, running at every
hour, and exercising a kind of cruel assault on those bodies of the
unfortunate females, upon which they are to learn their practice. But
will they learn it by this means? It is much to be doubted. It may
perhaps be granted, that men of a certain age, men past the slippery
season of youth, may claim the benefit of exemption from
impressions of sensuality, by objects to which custom has
familiarized them. But, in good faith, can this be hoped or expected
in the ungovernable fervor of youth? Can such a stoic insensibility be
imagined in a boy or young man, as that he can direct such his
researches by pawing and grabbling to the end of instruction only?
Must not those researches, humanly speaking, be made in such a
disorder of the senses, as to exclude the cool spirit of learning and
improvement? May he not lose himself, and yet not find what was
the occasion of losing himself? In short, granted, though it is surely
hard to grant, that the wretched women, admitted to this so falsely
called Charity, may not deserve much tender consideration; but in
what can the poor young pupils have deserved so ill of their parents
or guardians, as to be thus exposed to temptations so shockingly
indecent? What father, what mother, what considerate relation can
paint to himself a child, or charge of his, at an age so incapable of
resisting the power of sensual objects, as is that of youth, employed
in exploring such arcanums, and exploring them too in vain? It is
surely easier to guess the natural consequences, than to defend either
the subjecting youths to them, or the hoping any good from the
subjecting them. In short, even Dr. Smellie’s doll is a more laudable
method of instruction.

B u t besides this reason taken from the moral impossibility of

laying a timely foundation of practical knowledge in the male-sex, for
preferring women under the false charge of ignorance, to the so
unconsequentially boasted learning of the men, there remains a yet
stronger argument against the male-practitioners: an argument
furnished by nature herself, and of the which, every impartial
reader’s own feelings will in course render himself the judge.

N a t u r e has to all animals, from the man down to the lowest

insect, to all vegetables, from the cedar to the hyssop, to all created
beings, in short gives what is respectfully necessary for them. Nor
can it without the grossest absurdity be imagined, that this tender
universal parent, or call her by a yet more sacred name, the divine
providence, would have failed women in a point of so great
importance to them, as that of the ability to assist one another, in
lying-in, at the same time, that she has given them so strong and so
reasonable a sympathy for those of their sex in that condition? Can it
be thought that nature, so vigilant, so attentive, to the production of
fresh generations, through all beings, should have been deficient or
indifferent as to women, her favourite work, the friend, the ornament
of human kind? And so she must have been, if she had left her in the
necessity of recourse to others than those of her own sex, in whom
there exists so sensibly a superior aptitude for tending, nursing,
comforting and relieving the sick, that even the men themselves, in
their exigences of infirmities, can hardly do without them. But to say
the truth, and as I have before remarked, nature has been even
liberal in her accomplishments of those of the female sex for this
office. Not content with giving them a heart strong imprinted with a
particular sympathy for their own sex, on this occasion, a sympathy,
which for its tenderness, has some resemblance or affinity to the
instinctive love or storge that parents have for their children; she has
also bestowed on them a particular talent, both for the manual
function in the delivery of women, and for all the concomitant
requisites of their aid during the time of their lying-in: a talent in
short, which may even be felt, without the necessity of definition or
proof, to be superior to any possible attainment of the men in that
art, though they should have sacrificed hecatombs of pregnant
rabbits, or have brooded over thousands of coveys of eggs in their
search of excellence in it. To say nothing of a certain softness,
flexibility, and dexterity of hand, palpably denied to the men, there
is, both in the management of the manual operation, and in the
attendance due on those occasions, a quality in which the women,
generally speaking, excel the men, and that is, patience, a quality
more essential, more indispensable than can well be imagined. For
on patience it is, that the salvation of both mother and child often
depend; whether that patience is considered in the so needful point
of predisposing the passages, or of waiting, without however over-
waiting, the critical efforts of nature in the expulsion of her burden.
Now nothing is more certain, than that nature, who to woman has in
general given all that vivacity and quickness of spirit, which seems
incompatible with the phlegmatic quality of patience, has, as if she
had purposely meant an exception favourable to her darling end, the
propagation of beings, especially the human one, bestowed on the
female sex, such a remarkable assiduity and diligence in aid of
women’s labors, as are rarely to be seen in men, and when seen,
appear rather forced than naturally constitutional to them. Women,
in those cases, have more bowels for women: they feel for those of
their own sex so much, that that feeling operates in them like an
irresistible instinct, both in favor of the pregnant mother and of the
child. Thence it is, that a woman-practitioner will employ, without
stint, or remission, all that is necessary to predispose the passages,
for the least pain, and the greater safety; she will patiently, even to
sixteen, to eighteen hours, where an extraordinary case requires so
extraordinary a length of time, keep her hands fixedly employed in
reducing and preserving the uterus in a due position, so as that she
may not lapse the critical favorable moment of extraction, or of
assisting the expulsive effort of nature: and what man is there, can it
be imagined, would have endurance enough to remain so long in a
posture, the very image of which, in one of his sex, is so nauseating
and so revolting, to say nothing of the want of that pliability and
dexterity of management of the fingers, on those occasions, so
necessary, and so uncommon in the men, especially in that very age,
when their practice should be supposed the greatest.
 I t is then in those cases where nature is slow, as she sometimes is,
in her operation, and often so, for the greater good of the patient, so
conformed perhaps, that a quicker expulsion would only destroy her,
that the midwife, not only uses all patience consistent with safety of
life to the mother especially, but inculcates patience to her suffering
charge. Whereas the men, from their natural impatience, or from
whatever other motives their precipitation may arise, having those
infernal iron and steel instruments at hand, are but too often
tempted to make use of them, not only without necessity, but against
all the indications of nature, pleading for a just indulgence to her of
her own time in her own work. In vain then do too many of them
declaim as loudly as can be wished, or as the thing deserves, against
all recourse to instruments, but in extremities which, they pretend,
justify them. In the first place, those extremities are often the fault of
deficient and unskilful practice. The precious moments of the
assistence due to nature have been lapsed, or there has been some
failure of preliminary treatment; or what is worse yet, extremities are
rashly taken for granted when they are not existing.

H e r e, in the history of one single woman, I give the history

probably of thousands.

A h e a l t h y woman, about twenty five years of age, and

remarkably robust, was in labor of her second child. Her first had
come in that natural smooth way, as had given the same man-
midwife, who was now to lay her again, not the least trouble, as often
happens. In this second labor, however, the head of the child stuck in
the passage; and was so far advanced, that the Doctor told her,
whether in jest or earnest I cannot say, that he could discern the
color of its hair. Her pain, though extremely great, had not however
hindered her observing the Doctor rummaging for his instruments;
her frightful apprehension, of which, she had all the reason to
imagine, did not a little contribute to retard her throws. She taxed
him with his intention to use them, and he did not deny it. Upon this
she used the most moving fervorous entreaties for a respite of
execution; but all in vain; he told her, with a resolute tone, that he
knew surely better what was for her good than she did, that he had
even already waited longer than he could justify; and that her life was
absolutely desperate if the child was not instantly extracted, of the
which being dead, he was sure from many incontestable symptoms.
Her thorough confidence in a man, whom she had often heard
declaim vehemently against the use of instruments unless in
extremities, and which she understood in the most literal sense,
without considering, or perhaps knowing that, on too many
occasions, nothing is so different as words and actions; her thorough
confidence in him, I say, joined to a natural love of life, and to her
present feelings of exquisite pain, determined her to an
acquiescence. The fatal instrument was struck into the brain-pan of
the child, who at the instant gave the lie to the first part of the
Doctor’s asseveration as to its death, by such a strong kick inwards as
had almost killed her, and convinced her not only of its being alive
but lively. This did not, you may be sure, add to her belief of the
second part of his averment, that waiting any longer for the
operation of nature, would infallibly have been her death. It might be
so: yet surely there are strong reasons for concluding, that a little
more patience might have saved a fine boy, and yet not have
destroyed, or even hazarded the destroying the mother, whose life is
certainly the preferable object. But how cruel to state the dreadful
alternative where it does not exist! And how easy, in the presumption
of that alternative, to extort the dreadful consent from a weak
woman, yet more weakened by her condition, and naturally
determined by her present feelings, to embrace the appearance of an
immediate relief, presented to her in the form of salvation of life!
However, scenes similar or a-kin to this, may, without breach of
charity, be presumed too frequent, especially under those superficial
men-midwives, whom the facility of forming, in the manner they are
generally formed, renders so suspicious as to their ability, and who
for so many reasons, both of nature and interest, are but too liable to
the murderous want of that patience, for which the women are but
the more remarkable in this case, for their not being perhaps so
capable of it in any other. But here their duty is even their nature; as
if in so capital a point, she would trust it to nothing but herself.

I f it should be here to this objected that the women may, through

that very spirit of patience, wait too long, or overstay the time of
saving the patients life, for want of calling in proper assistence; I
have already implicitly obviated this objection, by remarking before,
that a true thorough midwife, from her quickness of apprehension,
and knowledge of the danger, will ever be readier to call in the
assistence and advice of a physician, than the common men-
midwives, who are ever in proportion to their ignorance the more
rash, the more fearless, and consequently averse to calling in that
help, of which they will be ashamed to confess their want, and thus
cruelly, though with impunity, lose the opportunity of others
endeavouring at least to repair those damages, of which themselves
are oftenest the authors. Now a midwife has no such shame; she
pretends to no extraordinary skill in physic or surgery; she knows
her art, and will not presume to transgress its bounds; she would
think herself accountable if she did: and even that very tenderness
and sensibility, upon which nature has founded her patience, will
make her cautious how she pushes that patience too far. She may
easily see, feel and discern those cases in which nature calls the
physician in aid to the midwife; nature, who seems to have placed
such boundaries between those professions, as nothing but interest,
presumption, or ignorance of nature, could ever render their union
in one person supposable: tho’ the quality of physician may not
indeed exclude that of the surgeon, but rather implies, at least, the
theory of surgery. For I presume anatomy is the great basis of true
rational physic, though it can very little assist practical midwifery,
which depends so much upon purely manual operation, and needs
only a sufficient general idea of the structure of the sexual parts in
woman, the conceptacle, and passages of the delivery.

T h i s is so true, that any impartial observer of the male and

female practitioners in midwifery, will easily distinguish the
characteristic difference of the sexes, in their respective manner of
operation.

I n the men, with all their boasted erudition, you cannot but
discern a certain, clumsy untowardly stiffness, an unaffectionate
perfunctory air, an ungainly management, that plainly prove it to be
an acquisition of art, or rather the rickety production of interest
begot upon art.

I n women, with all their supposed ignorance, you may observe a

certain shrewd vivacity, a grace of ease, a handiness of performance,
and especially a kind of unction of the heart, that all evidently
demonstrate this talent in them to be a genuine gift of nature, which
more than compensates what she is supposed to have refused them,
in depth of study, though even of that they are not so unsusceptible,
as some men detractingly think; and in midwifery, most certainly
they attain all that they need of learning to perfect them, with a
facility the greater for nature, having collaterally endowed them with
an organization of head, heart and hand, obviously adapting them to
this her most capital mystery. This will be denied by none who have
any regard for truth, and who do them justice, as to the keenness of
their apprehension, as to that simpathizing sensibility which supplies
them with the needful fund of patience, and tender attention; and as
to that peculiar suppleness of the fingers, as well as slight of hand, in
a function which rather exacts a kind of knack or dexterity, than
mere strength, of which they have also a competency. Nor can it be
quite without weight, that the midwives, besides their personal
experience, being sometimes themselves the mothers of children,
have a kind of intuitive guide within themselves, the original organ of
conception, itself pregnant, in more cases than that, with a strong
instinctive influence on the mind and actions of the sex; an influence
not the less certainly existing, for its being undefinable and
unaccountable, even to the greatest anatomists[4].

T h e men, it will be said, have many or all of these qualifications,

except indeed the last. Granted that they have: but how very few are
there of the men that possess the most essential ones to a degree
comparable to that of the women: or rather not so imperfectly, as
that all their boasted skill in literary theory and anatomy, cannot
supplement or atone for the deficiency? Nor theory, nor all the books
that ever were written on that subject from the divine Hippocrates,
who understood so much of physic, and so little of midwifery, down
to Dr. Smellie, who is so great a man in both, will ever amount to so
much as the practical experience of a regular bred midwife.

A s to that superior skill of the men in anatomy which is sounded

so high, against the women, I shall not imitate the men in their want
of candor towards the female-sex in their availing themselves of false
arguments. I will not then take the benefit of the slight opinion which
Celsus and Galen had of the depths of anatomy; they who contented
themselves with a gross superficial notion of the principal viscera. I
will not even desire to countenance that contempt by the example of
that great philosopher Mr. Lock, the intimate friend, and even the
counsellor of the British Esculapius Sydenham, who paid a great
deference to his physical knowledge; and yet this very Mr. Lock
wrote an ingenious treatise (though not published by him) upon the
insignificance of the refinements of anatomy in the practice of
physic. Neither will I here insist on the absurdities into which even
the greatest anatomists have fallen; as for example, Pecquet, the
famous discoverer of the thoracic duct in the human body, who
nevertheless adopted so extravagant a notion, as that digestion of
food ought not to be promoted by exercise, but by drinking
spirituous liquors, a practice to which himself fell a victim, dying
suddenly at the anatomical theatre. It is only for those who have a
false cause to defend to shut their eyes against those truths which
seem against them. Those on the contrary who defend purely the
truth, know that one truth cannot hurt or exclude another truth, and
that all truths may very well coexist. It may be true that anatomy,
though it does not give the nature of the elementary composition of
parts intrinsic and too minute for the human sense, since a new
incision only presents a new surface, much conduces however to
ground the student in mechanical principles of great assistence to
him in practice, of which they are doubtless the most solid
foundation: yet that truth is not incompatible with another quite as
much a truth, that midwifery can have no occasion but for a general
notion of the configuration of those parts upon which it is exercised.
A midwife, for example, may be a very safe and a very good one,
without knowing whether the uterus is a hollow muscle, or purely a
tissue of membranes, arteries and veins: but if that ascertainment is
necessary, she must wait for it till the anatomists have settled among
them that point, which, like many other capital points of anatomy, is
not however yet done. In short, once more, a woman in labor
requires a midwife to lay her, not an anatomist to dissect her, or read
lectures over the corpse, he will be most likely to make of her, if he
depends more on the refinements of anatomy, than on the dexterity
of hand, and the suggestions of practical experience and common
sense.

I f then, there are who can examine things fairly and with a sincere
desire of determining according to the preponderance of reason, they
cannot but on their own sense of nature, on their own feelings, in
short, discern that no ignorance, of which the women are
undistinguishingly taxed, can be an argument for the men’s
supplanting them in the practice of midwifery, on the strength of that
superiority of their learning, so rarely not perfectly superfluous, and
often dangerous, if not even destructive both to mother and child.
Consult nature, and her but too much despised oracle common
sense; consult even the writings of the men-midwives themselves,
and the resulting decision will be, that great reason there is to
believe, that the operation of the men-practitioners and
instrumentarians puts more women and infants to cruel and
torturous deaths, in the few countries where they are received, than
the ignorance of the midwives in all those countries put together
where the men-practitioners are not yet admitted, and where, for the
good of mankind, it is to be hoped they never will.

I h a v e here said few countries have hitherto countenanced men-

midwives. That I presume is too notorious to require proof: for even
those Saracen or Arabian physicians, Avicen, Rhazes, &c. who, by the
by, are little more than servile translators or copists of the Grecian
ones, wrote only theoretically in quality of physicians; for it does not
appear that they ever practised midwifery themselves, nor ever got
the practice of it by men introduced into their countries. Among the
Orientals there is no such being known as a man-midwife; that
refinement of real barbarism, under the specious pretext of
humanity, is happily unknown to them. But if it should be said, that
the jealousy so constitutional to the inhabitants of the warmer
climes, has a share in the exclusion of men-practitioners; the women
have, at least in that point, a weakness to thank for its production to
them of so great a good, as the greater safety of their persons and
children, in that capital emergency of their lying-in. For, after all, the
art of midwifery is, in the hands of men, like certain plants, which, by
dint of a forcing culture, exhibit more of florish, or a broader
expansion; but besides ever retaining a certain exotic appearance,
they never come up to the virtue of those spontaneously growing in
the full vigor of a soil of nature’s own choice for them. Art may often
indeed improve nature, but can never be a supplement to her, where
she is essentially wanting. Deep learning may, in very extraordinary
cases perhaps, repair the errors, or assist the deficiencies of the
manual function, but the deepest learning will never bestow the
manual function, nor indeed can in the same person exist, but at the
expence of the manual function, which must have been in some
measure neglected for it. And yet the greatest practical skill that any
man can with the utmost labor and experience acquire, will hardly
ever equal the excellence in it of the women, Great Nature’s chosen
instruments for this work: an excellence by them attained with scarce
any learning at all, or at least of that abstruse theoretical sort, on
which the men make their superiority principally depend.

B u t that I may not herein be taxed of maintaining any thing that

has only the air of a paradox, or of begging the question, I shall
implicitly, in the course of my answer to the following objection,
endeavor to remove any remaining doubt on this head.
 O b j e c t i o n the Eleventh.
I n like manner, as there are particular parts of the human body
which have their appropriate undertakers or protectors under their
proper distinctive names, as oculists, dentists, and corn-cutters, who
by making respectively one part their particular care and study,
arrive at a greater perfection, at least in the practical operations on it,
than regular physicians or surgeons, whose object is the whole fabric;
Why, by parity of reasoning, should not the men-practitioners in
midwifery be preferable to the midwives, since a man has to his
manual function superadded a theory superior to that of the women,
who, it is confessed, stand sometime in need of calling in the
physician to their assistence? As a man then will have laid in a stock
of medical knowledge, peculiarly adapted to the exigencies and
disorders incident to women during their pregnancy and lying-in, he
must consequently excel the midwife, or the physician singly
considered; he who with so much greater convenience will have
united in one person both their faculties, besides that of the surgeon.

ANSWER.

T h a t certain parts of the human body enjoy the protection of

practitioners, who respectively devote themselves to their service, I
confess. Such appropriations may also be beneficial, at least, to the
practitioners. I can even conceive, that a professed dentist may clean,
scale, and draw teeth, or an oculist couch a cataract, better than
either a physician or surgeon. These may in their respective practice
be excelled by those partial artists. But I much doubt, even as to
these, whether their trusting too much to that partial excellence,
does not sometimes do more mischief than good, for want of duly
consulting the relation of such parts to the universal fabric, of which
physicians and surgeons must be so much better judges. Galen does
not appear in contradiction to common sense, where he observes,
that to rectify a disorder of the eye, the head must be rectified, which
cannot well be done without rectifying the whole body. In
confirmation of which, I once myself knew a gentleman, whom a
professed oculist, at Paris, assured of the loss of his eyes being
infallible; and who upon his despondingly consulting a regular
physician, was by him as positively assured, that those very
condemned eyes might be saved by a proper regimen. The gentleman
happily believed him, and his eye-sight was not only saved, but
perfectly restored.

A n o t h e r instance of the like nature occurs to me, which seems

applicable to the dentist, and which I quote here from a translation
of the learned and ingenious Dr. Huxham’s observations on the
constitution of the air.

“M a n y years ago I knew a gentleman of a hale, robust habit of

body, who, from being too much addicted to the drinking of brandy,
fell into a violent jaundice, from which however he would have
recovered well enough, would he have conformed himself to the
advice of his physicians: but he on the contrary, because his gums
were very apt to bleed, and his teeth stunk from the scorbutic taint,
put himself into the hands of an ignorant pretender to physic for the
cure of these inconveniencies. This fellow immediately set about
scaling his teeth, and rubbing his gums with his famous teeth-
powder, till at last, by perpetually fretting and irritating the loose
texture, he brought on such a hemorrhage, that baffled all the stiptics
that could be invented by the most expert surgeons, and continuing
to spout forth in small streams from the little arteries of the gums,
which were now every where divided: in the space of sixteen hours
the poor man died through mere loss of blood.”

T h e s e instances are however only adduced to justify that doubt

which I expressed of these partial artists being always to be
beneficially consulted in those local affections, to which their talent
is supposed exclusively appropriated.

C o r n - c u t t e r is indeed a homely plain English term, but if the

teeth give from the Latin the appellation of dentist, as the eye that of
oculist, what name, taking it from the part in question, will remain
for that language, to give the men-practitioners of midwifery, in
substitution to that hermaphrodite appellation, that absurd
contradictory one in terms, of man-midwife, or to that new-fangled
word accoucheur, which is so rank and barefaced a gallicism? But let
what name soever be given them, it can hardly be too burlesque an
one, considering the gross revolting impropriety of men, addicting
themselves to a profession naturally so little made for them.

P a i n t to yourself one of these sage deep-learned Cotts, dressed

for proceeding to officiate[5], and presenting himself with his pocket-
nightgown, or loose washing wrapper, a waistcoat without sleeves,
and those of his shirt pinned up to the breasts of his waistcoat; add
to this,[6]fingers, if which not the nicest paring the nails will ever cure
the stiffness and clumsiness; and you will hardly deny its being
somewhat puzzling, the giving a name to such an heteroclite figure?
Or rather can a too ludicrous one be assigned it?

T h o s e however who will consider this grave Doctor in his

margery field-uniform, this ridiculous piece of mummery, in a light
of seriousness, such as the matter perhaps more justly deserves,
especially combining with all the rest, the idea of his crotchets,
forceps, and the rest of his bag of instruments, may think he less
resembles a priestess of Lucina, than the sacrificer, in a surplice, with
his slaughtering-knife, to one of those heathen deities whose horrid
worship required human victims, which the poor lying-in women but
too nearly resemble.

B u t whether or not, in imitation of the dentist, or oculist, he

receives his title from the particular part he has taken under his
protection, so much is certain, that the same arguments, which
militate for those partial artists claiming their respective
departments of the human body, will not avail the man-midwife. An
oculist, a dentist, a corn-cutter, have no operations to perform but
those of which disorders equally incident to both sexes are the object.
There is nothing in their practice repugnant to the nature of the
male-sex, nor to that reasonable decency, which only requires that no
sacrifices of it should be made in vain, or at least not made to no
better a purpose than to increase at once the danger and the pain of
both mother and child, in whose favor it is sacrificed, as it may be
clearly proved to be oftenest the case. But of the chirurgical part of
the man-midwife’s pretention, I reserve to treat after considering
him in the capacity of a physician; in which a man may indeed be
wanted, but in that of surgeon never, or at least so very rarely, as not
to atone for the dangers which attend the men forming themselves
into a set under the name of men-midwives.

W h e r e there is no complication of any collateral disorder with

the gestation and parturition of women, it is even a jest for men to
pretend the necessity of any study or practice to which women may
not arrive, and even much excel them.

B u t where there exists the case of a singular constitution, or of

symptoms declarative of other help being necessary than just the
common one, that quickness of discernment, that peculiar
shrewdness of the women, in distinguishing what is relative to their
art from what is foreign from it, gives them the alarm in time, and if
they have a just sense of their duty, or but common sense, they must
know that such disorders cannot be partial, cannot therefore be
considered as they are by the man-midwife, as subordinate to his
particular province, relative as they are to the whole fabric or system.
All partial practice then is here absolutely out of the question, and
now what help can, consistently with good sense, be expected from a
man-midwife, who, under a natural impossibility of ever acquiring
the female dexterity in the manual operation, cannot however, be
supposed to attain even that imperfect degree of skill, without
sacrificing to the endeavours at it the time and pains in study and
practice, which are requisite to form the able physician?

B u t, in fact, the men, that is to say, those of that sex who have the
best understood all the refinements of anatomy, all the variety of
female distempers, never that I can learn, attempted to invade the
practical province of midwifery. The immortal Harvey, Sydenham,
the great Boerhave, Haller, and numbers of others who have written
so usefully upon all the objects of midwifery, have never pretended
or dropped a hint of the expedience of substituting men-midwives to
the female ones. They contented themselves with lamenting the
ignorance of some midwives, from which has been drawn a very just
inference of the necessity of their being better instructed; but even
those great men never chose the character of practitioners
themselves, nor probably would have thought it any detraction from
their merit to have it said, they might make a bad figure in the
function of delivering a woman.

W h o e v e r then will consider but how the common run of men-

midwives actually are and must be formed, and assuredly the
number of exceptions to the general insufficiency cannot oppose the
inference, must allow that, where a woman has distempers collateral
to her pregnancy, with which they must also become dangerously
complicated, she must expose herself to the utmost hazard, in any
confidence she may place in a man-midwife.

T h e truth is, that most of the dangerous lyings-in are so far from
being likely to be relieved by a man-midwife, that it is often to the
having relied upon his medical judgment, and especially to his
manual skill they are owing. But of the first only it is we are now here
speaking.

T h e women captivated by that assiduity of the men-midwives, of

which they only fail when they are not paid or likely to be paid, in
some form or other, up to the value they set upon themselves, lightly
take for granted, that, as men, they are also capable physicians. It is
enough, in short, for these practitioners not to be women; for the
women to think they can prescribe for them in all disorders. A
mistake this, often big with the utmost danger to them.

T h e men-midwives, in general, have never, at the most, carried

their studies beyond the disorders commonly incident to pregnant
women: the knowledge of all the other possibly collateral ones, is
what even the least modest of them will hardly claim, unless to the
profoundly ignorant, and is in fact scarce less than impossible to one
who has applied himself essentially to the manual function. In such
cases the ignorance of a midwife can hardly be greater than that of
the men-practitioners, and must be less dangerous from her less of
pretention. Her consciousness of her own want of sufficient light,
will engage her readily to state the exigency to some able and
experienced physician, whom she must allow, in such cases, to be her
superior judge: whereas the other, the man-midwife, acknowledges
no greater authority than that with which he is pleased to invest
himself. He stands, in virtue of a distinct business, and a business for
which he never was made, of a sudden the self-constituted sovereign
dictator and inspector-general of all female disorders whatsoever,
where the woman is with child, that is to say, where the case is only
thereby rendered much the more nice and difficult, and, not rarely,
does he continue under the same pretext, to extend his practice to
where there is no pregnancy at all in the case. And yet ask him for his
titles, they are all implicitly dependent on or subordinate to that
same midwifery, for which he is so naturally unqualified, even if a
due study and exercise of it would permit those avocations, that
would contribute to accomplish him in the so necessary general
knowledge of physic. But indeed why need he acquire it, since it is so
commonly taken for granted, or that he is believed upon his own
word, especially if he is backed with a diploma, for form’s sake, that
may have cost him little or nothing of medical study, or indeed of any
thing but the amount of the fees for it?

Y e t how serious, how important is it for women, if they tender

their own lives, and that of the precious burthen of which they are
the depositaries, to make that distinction between the physician and
the midwife, which they seem so little to make! How little do they
consider, what nevertheless is strictly true, that a man can never at
the best be but an indifferent practitioner of midwifery, though he
may be an excellent one in physic; but that as bad a midwife as he
can be, he must be yet, if possible, a worse physician, if he attempts
to throw both professions into one, and exercise them jointly! They
are incompatible, from the justly presumable impossibility of one
man doing justice to the practice of the one, unless at the expence of
the study of the other: by which other, to obviate cavils, I repeat it, I
mean the general practice of physic, which comprehends the
speculative part of midwifery, as well as all other branches
understood to be the province of the physician. This distinction then
I make, because, as to the diseases purely incident to pregnant
women, experimental practice will rather assist the medical study of
them: and it is in that part only the men-midwives can make any
figure at all, and that not a superior one to midwives who are
regularly bred, and who have, in their favor, their excellence in the
manual function besides.
 O n c e more, in complicated cases, the most dreadful mistakes are
to be dreaded from those common-men-midwives, who so
groundlessly erect themselves into physicians on those occasions. A
purge, a venesection, or any other prescription injudiciously ordered,
may be the occasion proximate or remote of death to both mother
and child; yet a woman, at least, ought not to expect better from one
of these practitioners who, for the most part, has neither study nor
experience in general physic; nor more than a smattering of
anatomy, joined to the index-learning of dispensatories. Such a man-
midwife can never have thoroughly made himself master of the
course of the fluids, nor of the order of their circulation. Their
relation to the solids, and the efficacy of medicines upon both, can
hardly be sufficiently known to a man, who must have been too much
employed in trying to form a hand never to be formed, and in
attendances on the practice of his midwifery, to acquire those
collateral requisites for the effectual multiplication of his
professions.

Y e t this man void of knowledge, experience, observation, and, in

consequence, of physical ability, shall boldly decide on the
expedience of an internal remedy, of which he does not know the
power or operation; of a venesection, of which he can but guess at
the consequence; and of a narcotic, of which he is unaware of the
danger. In all which, observe, he may possibly sometimes be
tolerably right, in cases where there is no complication; that is to say,
in cases when a midwife, duly bred, is as sufficient as the best man-
practitioner. But then she is moreover not only quicker of
apprehension, as to danger, where the case appears complicated, but
readier to call in proper help where she discerns it to be above her
reach, and consequently above that of the man-midwife, who must
be equally or rather more at a loss, because his boasted theory will
serve only to puzzle him, or what is worse yet, since a shew must be
made of doing something, will most probably determine him
improperly, if not fatally, to random prescriptions, in points out of
his sphere of knowledge, or rote of practice.

M a n y a man who to-day undertakes prescribing for a fever, for a

fit, a convulsion in a lying-in woman, only because he appears in the
character of a man-midwife, would have been ashamed the day
before he had taken up that business to give himself out for a
physician. He would have been afraid of ordering any thing for her if
she was not his patient, as to lying-in, and would not, even after
assuming the profession of midwifery, perhaps order any thing for
the same woman, out of the time in which his office is supposed
necessary. This plainly proves, that many of those gentlemen are
weak enough to imagine, that the man-midwife implies the
physician, though the greatest physicians that ever were never
dreamt of such an absurdity, as that the physician implied the
midwife, whose master and instructor he rather is, in points highly
useful indeed at times to her profession, but in which that profession
does not consist.

I d o not however charge all the men-midwives with so much

modesty, as to confine their striking out of midwifery into physic, to
the women lying-in, or to the time of their lying-in, since there have
not been wanting some who, with equal ignorance, but superior
effrontery, have intrepidly hoisted, the standard of a general
knowledge of physic, and having originally insinuated themselves
into families in the character of men-midwives, have easily
maintained their ground in them afterwards on the foot of
physicians. A circumstance not much to be wondered at, considering
the endearment of such an office as that of a man-midwife, and the
ascendant it must serve to give them over the heads of families, even
in points where a midwife can have no shadow of pretention, for
interfering. In the mean time, let any one of sense or common
humanity consider but the consequences of this dangerous
admission of the sufficiency of a man-midwife in those complicated
cases, which require the consultation of a regular physician; to say
nothing, for the present, of the other objections already mentioned,
or which I shall hereafter more at large discuss, and the result must
be, to allow that the medical pretentions, or indeed any pretentions,
of these men-practitioners, cannot be too much discouraged, nor
confidence more misplaced than in them. For once that they may hit
the mark by chance, they will often take the part of the distemper
instead of that of the patient; they will do what they have only a gross
guess of being the right, not what they know to be so: and physic, at
best, but a conjectural science, must in them want even the common
grounds of conjecture.
 I n s t e a d then of the dangerous self-sufficiency of these complex
smatterers, you have in a plain midwife, supposing her regularly
bred, and duly qualified for her profession (for I am no more an
advocate for ignorance in the women than in the men) one, who,
being called in time, will duly consider, and observe the constitution
of the person that wants her assistence. If nothing appears
extraordinary, or out of the common-rules in her patient’s
constitution and conformation, she needs only lay down for her the
previous course of management, and as the hour of delivery
approaches predispose her properly: a point in which the men must
be grossly deficient, for want of that skill of prognostic inherent to
the women, from their particular delicacy and shrewdness in the
faculty of touching; upon which more depends than can be well
imagined. Wherever a case occurs to a midwife, so complicated as to
be above her reach, her interest, her reputation, her duty, all
conspire to prescribe to her a timely application to a regular
physician. She communicates her doubts or difficulties to him, who,
at the same time that he receives a just information from her of the
state of things, combines it with his own knowledge of the human
constitution. He does not confound, as the man-midwife does, ideas
so different as those of the manual operation, and the medicinal
prescription. The object of the physician, being the same as that of
the midwife, the prevention or alleviation of pain to the mother, and
the greatest safety to the mother and child, but preferentially that of
the mother; there is this advantage to both mother and child, that all
harshness of practice, all the violenter remedies will be as much
corrected as can be done, consistent with the safety of mother and
child, by the midwife’s tenderness, by which the physician will at the
same time be above the being misled into omissions of any thing
absolutely requisite. In short, on such occasions, they serve to
temper one another. A truly great physician will not disdain the
lights furnished him by her practical experience, and she knows the
bounds of her mechanical duty and profession too well, to interfere
with his superior intellectual province, in those points submitted to
it. A pragmatical man-midwife, on the strength of his miserable half-
learning, would think it a derogation from his character, to call in a
physician in supplement to his deficiency, of which he is always
ashamed, though indeed he has sometimes the excuse of himself not
knowing it. Then when a fatal accident has happened, under his
hands, against which, with more knowledge he might have guarded,
or which with less of presumption or dependence on himself he
might have prevented, by procuring previous or collateral advice; he
thinks himself abundantly acquitted by laying the blame on occult
causes. Even the great man-midwife, Mauriceau himself, has made
use of that trite exploded apology[7]: where he expressly says, “that a
sudden unexpected death of his patient was one of those
F A T A L I T I E S, that not all the human prudence can prevent.”

B u t that I may not here incur the least charge of unfairness, as if I

meant by this quotation any thing so absurd or unjust, as that in the
labors of pregnant women, as well as in other diseases unconnected
with them, there may not sometimes happen accidents impossible to
be foreseen, as well under the care of the best physician, called in by
the very best midwife, as under the most ignorant assuming man-
midwife, I shall here introduce another quotation from the same
Levret, that will especially shew the ladies, and all parties concerned,
to what an imaginary safety, so much, and even the very point sought
for, is sacrificed as is sacrificed, in preferring the men-practitioners
to the midwives. [8] “M. de la Motte says, that for the fifth time he laid
the wife of a glover of Valogne, the 16th of March, 1704; that the
woman was but an hour in her labor-pains, and that he delivered her
with all the facility imaginable; that he left her upon the couch till he
had given her some broth, after which he recommended her to the
care of the nurse, and went where his business called him. He adds,
that he had time but just to bleed two persons in the neighbourhood,
before he was fetched away in haste to see the patient he had just
laid, whom he found dead upon the bed. The cause of this death was
instantly manifest to him from the stream of blood, which ran about
the floor, and even penetrated to the apartment beneath, after
soaking through the bed itself, in which there remained clots of
blood of an extraordinary size.

“T h i s author adds, in the reflexions at the end of this

observation, that this delivery had been both more easy and more
expeditious than any this woman had precedently had: and he notes,
that these melancholic accidents are not without example, since
such ladies as the princess of ... and madam la Presidente de ——
with numbers of others, have, on the like occasion, undergone the
same fate, as her he here treats of. These are, according to him,
proofs that all human science and dexterity often cannot prevent the
like misfortunes, since these great ladies had been lain by the most
celebrated men-midwives.”

N o w I might here, without much probability of being

contradicted, aver, that where such accidents, said to happen so
frequently and inevitably, should happen under the hands of
midwives, there would be but one voice among the men-practitioners
and their credulous adherents, to impute them to the ignorance and
malpractice of the women. The plea of occult causes would be hooted
at in them, tho’ receivable, it seems, from the men.

N o t however to imitate what I condemn in them, a gross want of

candor to the women, of whom, by the by, the very best of the men-
practitioners have learnt all the laudable part of practice, I shall
allow that among those frequent examples, of sudden deaths upon
delivery, some few might perhaps be of those unaccountable
surprizes with which nature mocks human ignorance; but then it
must be allowed too, that not all of them admit of that favorable
solution. The truth is that nature, to those who have studied her
course, and watched her motions with a due spirit of practical
observation, hardly ever but gives warning enough to prepare proper
obviative methods. It is not here the place to enter into the
discussion of those deaths by sudden hemorrhage upon delivery, of
which I shall hereafter attempt to give a more satisfactory account, as
well as of the measures of prevention, than Levret. My end in the
preceding quotation is to show;

F i r s t, that by the confession of the men-midwives themselves,

the most fatal accidents frequently, and inevitably happen under
them in spite of all their science and dexterity!

S e c o n d l y, to offer to the reader a reflexion for himself to judge

of the validity of it, to wit, that, not only in the cases of the
hemorrhage, but in many others, where there is a complication of
disorders with the state of pregnancy and parturition, much of the
safety of mother and child must depend on that general medical
knowledge, to which the men-midwives have so little grounds of
pretention. Nor indeed, for the symptoms of necessity for resorting
to medical help, have they the same shrewd prognostic or acute sense
as the experienced women, who much sooner perceive the danger
before it is too late, and are neither with-held by a false shame, nor
by a criminal or senseless presumption, from calling in proper
assistence. Such at least has been and still is their practice in all ages,
and in all countries, where the matters of pregnancy and lyings-in
are committed to them. The great object of the man-midwife is to
impose so false a notion on his patient, as that his partial knowledge
is sufficient to every thing. The consequence of which is, that if he is
not too officious, too pragmatical, by way of ostentation of his art, in
common cases, that is to say, where there is no complication of
disorders, every thing may pass off tolerable well, till the crisis of
labor-pains. And in that crisis I defy him, with all his learning, to
equal the female skill and cleverness, not only for lessening the
sufferings of the patient, but for facilitating the happy issue of her
burden.

B u t where there is a complicated case, dependent on the

physician’s art, then the trusting to those men-dabblers in midwifery
is a folly that may be fatal to both mother and child, or, at the best,
the delivery will have been rendered more painful, more laborious,
more big with danger, for those precautions having been neglected,
which can be so little supposed to occur to the common run of men-
midwives in cases foreign from their rote of practice. Yet it is
precisely in those disorders collaterally contingent to pregnancy, and
no disorder does that state exclude, that the greatest skill and
knowledge of physic are required. Then it is, that not only the
preservation of the mother claims regard, and certainly the
preferable one, but even that of the child is no indifferent point. And
to save both, the state of the mothers constitution must be carefully
considered. Thus the combination of the disease with the pregnancy,
the due regard to the mother as well as that to the child, form a triple
object that takes in a compass of comprehension to which no
midwife will pretend, nor can be imagined to exist in the mere man-
practitioner of midwifery. Such a nicety of observation does not seem
to be the province of a manual operator, and indeed useless to him in
that character. And as he will be more likely to trust to conjectures,
which no sufficient grounds of study will have justified his
presuming to trust, he must oftener take the part of the disease than
of the patient. It is well if sometimes, disconcerted at the excess of a
danger of which he does not understand the origin or nature, he does
not, in default of the head, employ the hand, and engage the mother
in a premature or forced delivery of the child, to the imminent
hazard of the lives of both. Now comes the chirurgical operation in
play; and we shall now see, that the ingraftment of the surgeon upon
the midwife, deserves equally at least reprobation with that of the
physician.

B u t before I enter on this disquisition, I am to observe, that this

objection to the surgeon’s commencing midwife, does not in the least
attack the merit of that respectable body of men, the surgeons. No
one can honor their profession more than I do: I even readily grant,
that their skill in anatomy is of service to midwifery itself, into which
it throws a great light. It would be easy for me to name, if requisite,
several surgeons, who are not only an honor to their country, from
their excellence in an art so beneficial to mankind, but an ornament
to society, from their extensive humanity and charity. These, I am so
far from thinking, will hold themselves honored by the men-
midwives attempting to make a common-cause with them, that I
rather depend on their bearing witness on the part of the women in
this cause, which is indeed the cause of Nature, of that Nature which
they study so practically, consequently so usefully, and with which
they are so conversant. I am persuaded they can even furnish me
with arguments, from their superior store of knowledge, in
supplement to my deficiencies. The surgeons must look on these
professors of midwifery as a kind of amphibious beings, hard to
define, whose claim exhibits rather the deformity of a preternatural
excrescence, or wen growing out of the chirurgical art, than the
becomingness of a natural member of it. Most of the first founders of
this new sect of instrumentarians in this country were, or I am
greatly misinformed, neglected physicians, or surgeons without
practice, who in supplement to their respective deficiencies, greedily
snatched at the occasion at that time of a prevailing whim in France,
of employing men-midwives, with just such a rage of fashion, as
some of the ladies there prefer valet-de-chambres to waiting maids.
This novelty then appeared to practitioners despairing of business
enough in their own way, an excellent scheme for eking out their
scanty cloth with this bit of a border, of which by degrees they have
made to themselves a whole cloak. In short novelty joined, to the
much exagerated objections to perhaps a few insufficient midwives,
brought in and established a remedy yet worse than the disease.
Their success encouraged others; and now behold swarms of pupils
pullulating, and forming on the models before-mentioned. Thus two
or three maggots have produced thousands. Iron and steel are not
tender: and yet it was by the pretended necessity of resorting to
instruments made of these metals, that these out-casts of either
profession effectuated their introduction into a business so little
made for them. Then it was, that not with the least squinting view to
filthy lucre, but purely out of stark love and kindness to the women,
that these redressers of wrongs, armed with their crotchets, and
other weapons of death, took the field on the hardy adventure of
rescuing the fair sex out of the dreadful hands of the ignorant
midwives. But as to the validity of that plea of theirs, of the necessity
of employing instruments, I reserve to treat of it at large in its place
in my second part.

H e r e I shall only request the reader to remember, what has been

said of the indecent, superficial, and even cruel method of training
up pupils in this upstart profession. But if I was to add here my
having been credibly informed, that there are novices who watch the
distresses of poor pregnant women, even in private lodgings, where,
under a notion of learning the business, they make those poor
wretches, hired for their purpose, undergo the most inhuman
vexation, in a condition so fit to inspire compassion, and where those
scenes must be rather a school of brutality than of art: if I was to
urge, what from the great probability of the thing I firmly believe,
that more than one unhappy creature has fallen a victim to the
rudiments of these novices; that especially not long ago, one of them
in a hurry and confusion of presumption and ignorance, instead of
the after-birth from a woman, tore away, by mistake, her womb
itself, which occasioned, of all necessity, the poor creature’s dying in
unutterable agonies of torture: if I was yet to go farther and assert,
that even not one of the least eminent men-midwives pulled off the
arms of a child in his attempt to extract it, and very gravely laid them
upon the table; what would be replied to me? It would be said I had
invented these horrors, or forged such raw-head and bloody-bones
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookgate.com