Get the full ebook with Bonus Features for a Better Reading Experience on ebookmeta.

com

Deep Learning Patterns and Practices 1st Edition

Andrew Ferlitsch

https://ebookmeta.com/product/deep-learning-patterns-and-
practices-1st-edition-andrew-ferlitsch/

OR CLICK HERE

DOWLOAD NOW

Download more ebook instantly today at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Grokking Deep Learning MEAP Version 7 Andrew W Trask

https://ebookmeta.com/product/grokking-deep-learning-meap-
version-7-andrew-w-trask/

ebookmeta.com

Deep Learning in Data Analytics: Recent Techniques,

Practices and Applications 1st Edition Debi Prasanna
Acharjya
https://ebookmeta.com/product/deep-learning-in-data-analytics-recent-
techniques-practices-and-applications-1st-edition-debi-prasanna-
acharjya/
ebookmeta.com

Fluent C: Principles, Practices, and Patterns 1st Edition

Christopher Preschern

https://ebookmeta.com/product/fluent-c-principles-practices-and-
patterns-1st-edition-christopher-preschern/

ebookmeta.com

Third Person Singular A Mars Bahr Mystery Mars Bahr

Mysteries Book 1 K J Erickson

https://ebookmeta.com/product/third-person-singular-a-mars-bahr-
mystery-mars-bahr-mysteries-book-1-k-j-erickson/

ebookmeta.com
Quarkus in Action (MEAP V03) Martin Štefanko

https://ebookmeta.com/product/quarkus-in-action-meap-v03-martin-
stefanko/

ebookmeta.com

A Bluestocking for a Baron A Duke s Daughters The Elbury

Bouquet 3 1st Edition Arietta Richmond

https://ebookmeta.com/product/a-bluestocking-for-a-baron-a-duke-s-
daughters-the-elbury-bouquet-3-1st-edition-arietta-richmond/

ebookmeta.com

The Next Generation of Biomedical and Behavioral Sciences

Researchers Breaking Through 1st Edition And Medicine
Engineering National Academies Of Sciences
https://ebookmeta.com/product/the-next-generation-of-biomedical-and-
behavioral-sciences-researchers-breaking-through-1st-edition-and-
medicine-engineering-national-academies-of-sciences/
ebookmeta.com

Frontier Midwife Amanda Kate Bridges

https://ebookmeta.com/product/frontier-midwife-amanda-kate-bridges/

ebookmeta.com

Calculus on Heisenberg Manifolds AM 119 1st Edition

Richard Beals

https://ebookmeta.com/product/calculus-on-heisenberg-manifolds-
am-119-1st-edition-richard-beals/

ebookmeta.com
Collective Bargaining Among Photo Engravers in
Philadelphia Ordinary Methods Applied to an Occupation
Which Is Both an Art and a Manual Trade Charles Leese
https://ebookmeta.com/product/collective-bargaining-among-photo-
engravers-in-philadelphia-ordinary-methods-applied-to-an-occupation-
which-is-both-an-art-and-a-manual-trade-charles-leese/
ebookmeta.com
Andrew Ferlitsch

MANNING
Deep Learning Patterns and Practices
 Deep Learning
Patterns and Practices
ANDREW FERLITSCH

MANNING
SHELTER ISLAND
For online information and ordering of this and other Manning books, please visit
www.manning.com. The publisher offers discounts on this book when ordered in quantity.
For more information, please contact
Special Sales Department
Manning Publications Co.
20 Baldwin Road
PO Box 761
Shelter Island, NY 11964
Email: orders@manning.com

©2021 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in

any form or by means electronic, mechanical, photocopying, or otherwise, without prior written
permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks. Where those designations appear in the book, and Manning
Publications was aware of a trademark claim, the designations have been printed in initial caps
or all caps.

Recognizing the importance of preserving what has been written, it is Manning’s policy to have
the books we publish printed on acid-free paper, and we exert our best efforts to that end.
Recognizing also our responsibility to conserve the resources of our planet, Manning books are
printed on paper that is at least 15 percent recycled and processed without the use of elemental
chlorine.

The author and publisher have made every effort to ensure that the information in this book
was correct at press time. The author and publisher do not assume and hereby disclaim any
liability to any party for any loss, damage, or disruption caused by errors or omissions, whether
such errors or omissions result from negligence, accident, or any other cause, or from an usage
of the information herein.

Manning Publications Co. Development editor: Frances Lefkowitz

20 Baldwin Road Technical development editor: Al Krinker
PO Box 761 Review editor: Aleksandar Dragosavljević
Shelter Island, NY 11964 Production editor: Deirdre S. Hiam
Copy editor: Sharon Wilkey
Proofreader: Keri Hales
Technical proofreader: Karsten Strobaek
Typesetter: Gordan Salinovic
Cover designer: Marija Tudor

ISBN 9781617298264
Printed in the United States of America
 brief contents
PART 1 DEEP LEARNING FUNDAMENTALS ..........................................1
1 ■ Designing modern machine learning 3
2 ■ Deep neural networks 19
3 ■ Convolutional and residual neural networks 43
4 ■ Training fundamentals 70

PART 2 BASIC DESIGN PATTERN ..................................................119

5 ■ Procedural design pattern 121
6 ■ Wide convolutional neural networks 150
7 ■ Alternative connectivity patterns 183
8 ■ Mobile convolutional neural networks 209
9 ■ Autoencoders 250

PART 3 WORKING WITH PIPELINES ..............................................273

10 ■ Hyperparameter tuning 275
11 ■ Transfer learning 302
12 ■ Data distributions 325
13 ■ Data pipeline 346
14 ■ Training and deployment pipeline 386

v
 contents
preface xiii
acknowledgments xiv
about this book xv
about the author xx
about the cover illustration xxii

PART 1 DEEP LEARNING FUNDAMENTALS ................................1

1 Designing modern machine learning

1.1 A focus on adaptability 4
3

Computer vision leading the way 5 ■

Beyond computer vision:
NLP, NLU, structured data 6
1.2 The evolution in machine learning approaches 7
Classical AI vs. narrow AI 7 ■
Next steps in computer
learning 10
1.3 The benefits of design patterns 16

vii
viii CONTENTS

2 Deep neural networks

2.1 Neural network basics
19
20
Input layer 20 Deep neural networks 22 Feed-forward
■ ■

networks 23 Sequential API method 23 Functional API

■ ■

method 24 Input shape vs. input layer 24 Dense layer 25

■ ■

Activation functions 27 Shorthand syntax 30 Improving

■ ■

accuracy with an optimizer 30

2.2 DNN binary classifier 31
2.3 DNN multiclass classifier 34
2.4 DNN multilabel multiclass classifier 35
2.5 Simple image classifier 38
Flattening 38 ■
Overfitting and dropout 39

3 Convolutional and residual neural networks

3.1 Convolutional neural networks 44
43

Why we use a CNN over a DNN for image models 45

Downsampling (resizing) 45 Feature detection 46 ■

Pooling 48 Flattening 49
■

3.2 The ConvNet design for a CNN 50

3.3 VGG networks 54
3.4 ResNet networks 57
Architecture 58 ■
Batch normalization 63 ■
ResNet50 64

4 Training fundamentals
4.1 Forward feeding and backward propagation
70
71
Feeding 71 ■
Backward propagation 72
4.2 Dataset splitting 74
Training and test sets 74 ■
One-hot encoding 75
4.3 Data normalization 78
Normalization 78 ■
Standardization 80
4.4 Validation and overfitting 80
Validation 80 ■
Loss monitoring 84 ■
Going deeper with
layers 84
4.5 Convergence 86
4.6 Checkpointing and early stopping 88
Checkpointing 88 ■
Early stopping 90
4.7 Hyperparameters 91
 CONTENTS ix

Epochs 91 ■
Steps 92 ■
Batch size 94 ■
Learning rate 95
4.8 Invariance 97
Translational invariance 98 Scale invariance 104 ■

TF.Keras ImageDataGenerator 106

4.9 Raw (disk) datasets 108
Directory structure 108 CSV file 110 ■ ■
JSON file 111
Reading images 111 Resizing 114 ■

4.10 Model save/restore 116

Save 116 ■
Restore 116

PART 2 BASIC DESIGN PATTERN ........................................119

5 Procedural design pattern

5.1 Basic neural network architecture
121
123
5.2 Stem component 125
VGG 125 ■
ResNet 126 ■
ResNeXt 131 ■
Xception 132
5.3 Pre-stem 133
5.4 Learner component 134
ResNet 136 ■
DenseNet 139
5.5 Task component 141
ResNet 141 ■
Multilayer output 142 ■
SqueezeNet 145
5.6 Beyond computer vision: NLP 146
Natural-language understanding 146 ■
Transformer
architecture 148

6 Wide convolutional neural networks

6.1 Inception v1 152
150

Naive inception module 152 Inception v1 module 154 ■

Stem 157 Learner 157 Auxiliary classifiers 158

■ ■

Classifier 160
6.2 Inception v2: Factoring convolutions 161
6.3 Inception v3: Architecture redesign 163
Inception groups and blocks 165 Normal convolution ■
168
Spatial separable convolution 169 Stem redesign and ■

implementation 169 Auxiliary classifier 171

■
x CONTENTS

6.4 ResNeXt: Wide residual neural networks 172

ResNeXt block 172 ■
ResNeXt architecture 175
6.5 Wide residual network 176
WRN-50-2 architecture 177 ■
Wide residual block 178
6.6 Beyond computer vision: Structured data 179

7 Alternative connectivity patterns

7.1 DenseNet: Densely connected convolutional neural
183

network 185
Dense group 185 Dense block 188 DenseNet macro-
■ ■

architecture 190 Dense transition block 191

■

7.2 Xception: Extreme Inception 192

Xception architecture 193 Entry flow of Xception 195
■

Middle flow of Xception 198 Exit flow of Xception 200

■

Depthwise separable convolution 202 Depthwise ■

convolution 202 Pointwise convolution 203

■

7.3 SE-Net: Squeeze and excitation 203

Architecture of SE-Net 204 ■
Group and block of SE-Net 205
SE link 206

8 Mobile convolutional neural networks

8.1 MobileNet v1 210
209

Architecture 211 Width multiplier 212 Resolution

■ ■

multiplier 212 Stem 214 Learner 216 Classifier

■ ■ ■
218
8.2 MobileNet v2 219
Architecture 219 ■
Stem 220 ■
Learner 221 ■
Classifier 225
8.3 SqueezeNet 226
Architecture 227 ■
Stem 228 ■
Learner 228 ■
Classifier 231
Bypass connections 233
8.4 ShuffleNet v1 237
Architecture 237 ■
Stem 238 ■
Learner 238
8.5 Deployment 246
Quantization 246 ■
TF Lite conversion and prediction 247

9 Autoencoders
9.1
250
Deep neural network autoencoders 251
Autoencoder architecture 251 ■
Encoder 252 ■
Decoder 253
Training 254
 CONTENTS xi

9.2 Convolutional autoencoders 255

Architecture 256 ■
Encoder 257 ■
Decoder 258
9.3 Sparse autoencoders 259
9.4 Denoising autoencoders 260
9.5 Super-resolution 261
Pre-upsampling SR 261 ■
Post-upsampling SR 264
9.6 Pretext tasks 267
9.7 Beyond computer vision: sequence to sequence 270

PART 3 WORKING WITH PIPELINES ....................................273

10 Hyperparameter tuning
10.1 Weight initialization
275
277
Weight distributions 277 ■
Lottery hypothesis 278 ■
Warm-up
(numerical stability) 280
10.2 Hyperparameter search fundamentals 283
Manual method for hyperparameter search 284 Grid ■

search 285 Random search 286 KerasTuner 289

■ ■

10.3 Learning rate scheduler 291

Keras decay parameter 292 ■
Keras learning rate scheduler 292
Ramp 293 Constant step
■
294 Cosine annealing 295
■

10.4 Regularization 297

Weight regularization 297 ■
Label smoothing 298
10.5 Beyond computer vision 300

11 Transfer learning
11.1
302
TF.Keras prebuilt models 304
Base model 305 Pretrained ImageNet models for
■

prediction 306 New classifier 307

■

11.2 TF Hub prebuilt models 311

Using TF Hub pretrained models 312 ■
New classifier 314
11.3 Transfer learning between domains 315
Similar tasks 315 Distinct tasks 317 Domain-specific
■ ■

weights 319 Domain transfer weight initialization 321

■

Negative transfer 323

11.4 Beyond computer vision 323
xii CONTENTS

12 Data distributions
12.1 Distribution types
325
326
Population distribution 327 Sampling distribution ■
328
Subpopulation distribution 329
12.2 Out of distribution 330
The MNIST curated dataset 330 Setting up the ■

environment 331 The challenge (“in the wild”) 331

■

Training as a DNN 332 Training as a CNN 339 Image■ ■

augmentation 342 Final test 344 ■

13 Data pipeline
13.1
346
Data formats and storage 348
Compressed and raw-image formats 348 ■
HDF5 format 352
DICOM format 356 TFRecord format ■
358
13.2 Data feeding 364
NumPy 364 ■
TFRecord 366
13.3 Data preprocessing 368
Preprocessing with a pre-stem 368 ■
Preprocessing with TF
Extended 376
13.4 Data augmentation 381
Invariance 381 ■
Augmentation with tf.data 383 ■
Pre-stem 384

14 Training and deployment pipeline

14.1 Model feeding 388
386

Model feeding with tf.data.Dataset 392 Distributed feeding with ■

tf.Strategy 395 Model feeding with TFX 397

■

14.2 Training schedulers 404

Pipeline versioning 406 ■
Metadata 408 ■
History 410
14.3 Model evaluations 412
Candidate vs. blessed model 412 ■
TFX evaluation 416
14.4 Serving predictions 419
On-demand (live) serving 419 Batch prediction 422 TFX ■ ■

pipeline components for deployment 423 A/B testing 425 ■

Load balancing 427 Continuous evaluation 429

■

14.5 Evolution in production pipeline design 430

Machine learning as a pipeline 430 Machine learning as a CI/CD ■

production process 431 Model amalgamation in production 431

■

index 435
 preface
As a Googler, one of my duties is to educate software engineers on how to use machine
learning. I already had experience creating online tutorials, meetups, conference pre-
sentations, training workshops, and coursework for private coding schools and univer-
sity graduate studies, but I am always looking for new ways to effectively teach.
Prior to Google, I worked in Japanese IT as a principal research scientist for 20
years—all without deep learning. Almost everything I see today, we were doing in
innovation labs 15 years ago; the difference is we needed a room full of scientists and
a vast budget. It’s incredible how things have so rapidly changed as a result of deep
learning.
Back in the late 2000s, I was working with small structured datasets with geospatial
data from national and international sources all over the world. Coworkers called me
a data scientist, but nobody knew what a data scientist really was. Then came big data,
and I didn’t know the big data tools and frameworks, and suddenly I wasn’t a data sci-
entist. What? I had to scramble and learn the tools and concepts behind big data and
once again I was a data scientist.
Then emerged machine learning on big datasets, like linear/logistic regression
and CART analysis, and I hadn’t used statistics since graduate school decades ago, and
once again I was not a data scientist. What? I had to scramble to learn statistics all over
again, and once again I was a data scientist. Then came deep learning, and I didn’t
know the theory and frameworks for neural networks and suddenly I wasn’t a data sci-
entist. What? I scrambled again and learned deep learning theory and other deep
learning frameworks. And once again, I am a data scientist.

xiii
 acknowledgments
I would like to thank all those at Manning who helped throughout this process. Fran-
ces Lefkowitz, my development editor; Deirdre Hiam, my project editor; Sharon
Wilkey, my copyeditor; Keri Hales, my proofreader; and Aleksandar Dragosavljević,
my reviewing editor.
To all the reviewers: Ariel Gamino, Arne Peter Raulf, Barry Siegel, Brian R. Gaines,
Christopher Marshall, Curtis Bates, Eros Pedrini, Hilde Van Gysel, Ishan Khurana, Jen
Lee, Karthikeyarajan Rajendran, Michael Kareev, Muhammad Sohaib Arif, Nick
Vazquez, Ninoslav Cerkez, Oliver Korten, Piyush Mehta, Richard Tobias, Romit Sing-
hai, Sayak Paul, Sergio Govoni, Simone Sguazza, Udendran Mudaliyar, Vishwesh Ravi
Shrimali, and Viton Vitanis, your suggestions helped make this a better book.
To all Google Cloud AI staff who have shared their personal and customer insights,
your insights helped the book cover a broader audience.

xiv
Discovering Diverse Content Through
Random Scribd Documents
 6. Mrs. Greene’s will makes the five children equal beneficiaries.
In event of death of any of them the survivors share alike; and if all
should die the estate goes to their families, if any.
7. The sleeping-rooms of the Greenes are arranged thus: Julia’s
and Rex’s face each other at the front of the house; Chester’s and
Ada’s face each other in the centre of the house; and Sibella’s and
Mrs. Greene’s face each other at the rear. No two rooms
intercommunicate, with the exception of Ada’s and Mrs. Greene’s;
and these two rooms also give on the same balcony.
8. The library of Tobias Greene, which Mrs. Greene believes she
had kept locked for twelve years, contains a remarkably complete
collection of books on criminology and allied subjects.
9. Tobias Greene’s past was somewhat mysterious, and there
were many rumors concerning shady transactions carried on by him
in foreign lands.

First Crime
10. Julia is killed by a contact shot, fired from the front, at 11.30 P.
M.
11. Ada is shot from behind, also by a contact shot. She
recovers.
12. Julia is found in bed, with a look of horror and amazement on
her face.
13. Ada is found on the floor before the dressing-table.
14. The lights have been turned on in both rooms.
15. Over three minutes elapse between the two shots.
16. Von Blon, summoned immediately, arrives within half an hour.
17. A set of footprints, other than Von Blon’s, leaving and
approaching the house, is found; but the character of the snow
renders them indecipherable.
18. The tracks have been made during the half-hour preceding
the crime.
19. Both shootings are done with a .32 revolver.
20. Chester reports that an old .32 revolver of his is missing.
21. Chester is not satisfied with the police theory of a burglar, and
insists that the District Attorney’s office investigate the case.
 22. Mrs. Greene is aroused by the shot fired in Ada’s room, and
hears Ada fall. But she hears no footsteps or sound of a door
closing.
23. Sproot is half-way down the servants’ stairs when the second
shot is fired, yet he encounters no one in the hall. Nor does he hear
any noise.
24. Rex, in the room next to Ada’s, says he heard no shot.
25. Rex intimates that Chester knows more about the tragedy
than he admits.
26. There is some secret between Chester and Sibella.
27. Sibella, like Chester, repudiates the burglar theory, but
refuses to suggest an alternative, and says frankly that any member
of the Greene family may be guilty.
28. Ada says she was awakened by a menacing presence in her
room, which was in darkness; that she attempted to run from the
intruder, but was pursued by shuffling footsteps.
29. Ada says a hand touched her when she first arose from bed,
but refuses to make any attempt to identify the hand.
30. Sibella challenges Ada to say that it was she (Sibella) who
was in the room, and then deliberately accuses Ada of having shot
Julia. She also accuses Ada of having stolen the revolver from
Chester’s room.
31. Von Blon, by his attitude and manner, reveals a curious
intimacy between Sibella and himself.
32. Ada is frankly fond of Von Blon.

Second Crime
33. Four days after Julia and Ada are shot, at 11.30 p. m., Chester
is murdered by a contact shot fired from a .32 revolver.
34. There is a look of amazement and horror on his face.
35. Sibella hears the shot and summons Sproot.
36. Sibella says she listened at her door immediately after the
shot was fired, but heard no other sound.
37. The lights are on in Chester’s room. He was apparently
reading when the murderer entered.
 38. A clear double set of footprints is found on the front walk. The
tracks have been made within a half-hour of the crime.
39. A pair of galoshes, exactly corresponding to the footprints, is
found in Chester’s clothes-closet.
40. Ada had a premonition of Chester’s death, and, when
informed of it, guesses he has been shot in the same manner as
Julia. But she is greatly relieved when shown the footprint patterns
indicating that the murderer is an outsider.
41. Rex says he heard a noise in the hall and the sound of a door
closing twenty minutes before the shot was fired.
42. Ada, when told of Rex’s story, recalls also having heard a
door close at some time after eleven.
43. It is obvious that Ada knows or suspects something.
44. The cook becomes emotional at the thought of any one
wanting to harm Ada, but says she can understand a person having
a reason to shoot Julia and Chester.
45. Rex, when interviewed, shows clearly that he thinks some
one in the house is guilty.
46. Rex accuses Von Blon of being the murderer.
47. Mrs. Greene makes a request that the investigation be
dropped.

Third Crime
48. Rex is shot in the forehead with a .32 revolver, at 11.20 a. m.,
twenty days after Chester has been killed and within five minutes of
the time Ada phones him from the District Attorney’s office.
49. There is no look of horror or surprise on Rex’s face, as was
the case with Julia and Chester.
50. His body is found on the floor before the mantel.
51. A diagram which Ada asked him to bring with him to the
District Attorney’s office has disappeared.
52. No one up-stairs hears the shot, though the doors are open;
but Sproot, down-stairs in the butler’s pantry, hears it distinctly.
53. Von Blon is visiting Sibella that morning; but she says she
was in the bathroom bathing her dog at the time Rex was shot.
 54. Footprints are found in Ada’s room coming from the balcony
door, which is ajar.
55. A single set of footprints is found leading from the front walk
to the balcony.
56. The tracks could have been made at any time after nine
o’clock that morning.
57. Sibella refuses to go away on a visit.
58. The galoshes that made all three sets of footprints are found
in the linen-closet, although they were not there when the house was
searched for the revolver.
59. The galoshes are returned to the linen-closet, but disappear
that night.

Fourth Crime
60. Two days after Rex’s death Ada and Mrs. Greene are
poisoned within twelve hours of each other—Ada with morphine,
Mrs. Greene with strychnine.
61. Ada is treated at once, and recovers.
62. Von Blon is seen leaving the house just before Ada swallows
the poison.
63. Ada is discovered by Sproot as a result of Sibella’s dog
catching his teeth in the bell-cord.
64. The morphine was taken in the bouillon which Ada habitually
drank in the mornings.
65. Ada states that no one visited her in her room after the nurse
had called her to come and drink the bouillon; but that she went to
Julia’s room to get a shawl, leaving the bouillon unguarded for
several moments.
66. Neither Ada nor the nurse remembers having seen Sibella’s
dog in the hall before the poisoned bouillon was taken.
67. Mrs. Greene is found dead of strychnine-poisoning the
morning after Ada swallowed the morphine.
68. The strychnine could have been administered only after 11 p.
m. the previous night.
69. The nurse was in her room on the third floor between 11 and
11.30 p. m.
 70. Von Blon was calling on Sibella that night, but Sibella says he
left her at 10.45.
71. The strychnine was administered in a dose of citrocarbonate,
which, presumably, Mrs. Greene would not have taken without
assistance.
72. Sibella decides to visit a girl chum in Atlantic City, and leaves
New York on the afternoon train.

Distributable Facts
73. The same revolver is used on Julia, Ada, Chester, and Rex.
74. All three sets of footprints have obviously been made by
some one in the house for the purpose of casting suspicion on an
outsider.
75. The murderer is some one whom both Julia and Chester
would receive in their rooms, in negligé, late at night.
76. The murderer does not make himself known to Ada, but
enters her room surreptitiously.
77. Nearly three weeks after Chester’s death Ada comes to the
District Attorney’s office, stating she has important news to impart.
78. Ada says that Rex has confessed to her that he heard the
shot in her room and also heard other things, but was afraid to admit
them; and she asks that Rex be questioned.
79. Ada tells of having found a cryptic diagram, marked with
symbols, in the lower hall near the library door.
80. On the day of Rex’s murder Von Blon reports that his
medicine-case has been rifled of three grains of strychnine and six
grains of morphine—presumably at the Greene mansion.
81. The library reveals the fact that some one has been in the
habit of going there and reading by candle-light. The books that
show signs of having been read are: a handbook of the criminal
sciences, two works on toxicology, and two treatises on hysterical
paralysis and sleep-walking.
82. The visitor to the library is some one who understands
German well, for three of the books that have been read are in
German.
 83. The galoshes that disappeared from the linen-closet on the
night of Rex’s murder are found in the library.
84. Some one listens at the door while the library is being
inspected.
85. Ada reports that she saw Mrs. Greene walking in the lower
hall the night before.
86. Von Blon asserts that Mrs. Greene’s paralysis is of a nature
that makes movement a physical impossibility.
87. Arrangements are made with Von Blon to have Doctor
Oppenheimer examine Mrs. Greene.
88. Von Blon informs Mrs. Greene of the proposed examination,
which he has scheduled for the following day.
89. Mrs. Greene is poisoned before Doctor Oppenheimer’s
examination can be made.
90. The post mortem reveals conclusively that Mrs. Greene’s leg
muscles were so atrophied that she could not have walked.
91. Ada, when told of the autopsy, insists that she saw her
mother’s shawl about the figure in the hall, and, on being pressed,
admits that Sibella sometimes wore it.
92. During the questioning of Ada regarding the shawl Mrs.
Mannheim suggests that it was she herself whom Ada saw in the
hall.
93. When Julia and Ada were shot there were, or could have
been, present in the house: Chester, Sibella, Rex, Mrs. Greene, Von
Blon, Barton, Hemming, Sproot, and Mrs. Mannheim.
94. When Chester was shot there were, or could have been,
present in the house: Sibella, Rex, Mrs. Greene, Ada, Von Blon,
Barton, Hemming, Sproot, and Mrs. Mannheim.
95. When Rex was shot there were, or could have been, present
in the house: Sibella, Mrs. Greene, Von Blon, Hemming, Sproot, and
Mrs. Mannheim.
96. When Ada was poisoned there were, or could have been,
present in the house: Sibella, Mrs. Greene, Von Blon, Hemming,
Sproot, and Mrs. Mannheim.
97. When Mrs. Greene was poisoned there were, or could have
been, present in the house: Sibella, Von Blon, Ada, Hemming,
Sproot, and Mrs. Mannheim.
 When Markham had finished reading the summary, he went
through it a second time. Then he laid it on the table.
“Yes, Vance,” he said, “you’ve covered the main points pretty
thoroughly. But I can’t see any coherence in them. In fact, they seem
only to emphasize the confusion of the case.”
“And yet, Markham, I’m convinced that they only need
rearrangement and interpretation to be perfectly clear. Properly
analyzed, they’ll tell us everything we want to know.”
Markham glanced again through the pages.
“If it wasn’t for certain items, we could make out a case against
several people. But no matter what person in the list we may assume
to be guilty, we are at once confronted by a group of contradictory
and insurmountable facts. This précis could be used effectively to
prove that every one concerned is innocent.”
“Superficially it appears that way,” agreed Vance. “But we first
must find the generating line of the design, and then relate the
subsidi’ry forms of the pattern to it.”
Markham made a hopeless gesture.
“If only life were as simple as your æsthetic theories!”
“It’s dashed simpler,” Vance asserted. “The mere mechanism of a
camera can record life; but only a highly developed creative
intelligence, with a profound philosophic insight, can produce a work
of art.”
“Can you make any sense—æsthetic or otherwise—out of this?”
Markham petulantly tapped the sheets of paper.
“I can see certain traceries, so to speak—certain suggestions of a
pattern; but I’ll admit the main design has thus far eluded me. The
fact is, Markham, I have a feeling that some important factor in this
case—some balancing line of the pattern, perhaps—is still hidden
from us. I don’t say that my résumé is insusceptible of interpretation
in its present state; but our task would be greatly simplified if we
were in possession of the missing integer.”
Fifteen minutes later, when we had returned to Markham’s main
office, Swacker came in and laid a letter on the desk.
“There’s a funny one, Chief,” he said.
Markham took up the letter and read it with a deepening frown.
When he had finished, he handed it to Vance. The letter-head read,
“Rectory, Third Presbyterian Church, Stamford, Connecticut”; the
date was the preceding day; and the signature was that of the
Reverend Anthony Seymour. The contents of the letter, written in a
small, precise hand, were as follows:

The Honorable John F.-X. Markham,

Dear Sir: As far as I am aware, I have never betrayed a
confidence. But there can arise, I believe, unforeseen circumstances
to modify the strictness of one’s adherence to a given promise, and
indeed impose upon one a greater duty than that of keeping silent.
I have read in the papers of the wicked and abominable things
that have happened at the Greene residence in New York; and I
have therefore come to the conclusion, after much heart-searching
and prayer, that it is my bounden duty to put you in possession of a
fact which, as the result of a promise, I have kept to myself for over a
year. I would not now betray this trust did I not believe that some
good might possibly come of it, and that you, my dear sir, would also
treat the matter in the most sacred confidence. It may not help you—
indeed, I do not see how it can possibly lead to a solution of the
terrible curse that has fallen upon the Greene family—but since the
fact is connected intimately with one of the members of that family, I
will feel better when I have communicated it to you.
On the night of August 29, of last year, a machine drove up to my
door, and a man and a woman asked that I secretly marry them. I
may say that I am frequently receiving such requests from runaway
couples. This particular couple appeared to be well-bred dependable
people, and I concurred with their wishes, giving them my
assurances that the ceremony would, as they desired, be kept
confidential.
The names that appeared on the license—which had been
secured in New Haven late that afternoon—were Sibella Greene, of
New York City, and Arthur Von Blon, also of New York City.

Vance read the letter and handed it back.

“Really, y’ know, I can’t say that I’m astonished——”
Suddenly he broke off, his eyes fixed thoughtfully before him.
Then he rose nervously and paced up and down.
 “That tears it!” he exclaimed.
Markham threw him a look of puzzled interrogation.
“What’s the point?”
“Don’t you see?” Vance came quickly to the District Attorney’s
desk. “My word! That’s the one fact that’s missing from my
tabulation.” He then unfolded the last sheet and wrote:

98. Sibella and Von Blon were secretly married a year ago.

“But I don’t see how that helps,” protested Markham.

“Neither do I at this moment,” Vance replied. “But I’m going to
spend this evening in erudite meditation.”
 CHAPTER XXIV.
A Mysterious Trip
(Sunday, December 5)

The Boston Symphony Orchestra was scheduled that afternoon

to play a Bach Concerto and Beethoven’s C-Minor Symphony; and
Vance, on leaving the District Attorney’s office, rode direct to
Carnegie Hall. He sat through the concert in a state of relaxed
receptivity, and afterward insisted on walking the two miles back to
his quarters—an almost unheard-of thing for him.
Shortly after dinner Vance bade me good night and, donning his
slippers and house-robe, went into the library. I had considerable
work to do that night, and it was long past midnight when I finished.
On the way to my room I passed the library door, which had been left
slightly ajar, and I saw Vance sitting at his desk—his head in his
hands, the summary lying before him—in an attitude of oblivious
concentration. He was smoking, as was habitual with him during any
sort of mental activity; and the ash-receiver at his elbow was filled
with cigarette-stubs. I moved on quietly, marvelling at the way this
new problem had taken hold of him.
It was half past three in the morning when I suddenly awoke,
conscious of footsteps somewhere in the house. Rising quietly, I
went into the hall, drawn by a vague curiosity mingled with
uneasiness. At the end of the corridor a panel of light fell on the wall,
and as I moved forward in the semidarkness I saw that the light
issued from the partly open library door. At the same time I became
aware that the footsteps, too, came from that room. I could not resist
looking inside; and there I saw Vance walking up and down, his chin
sunk on his breast, his hands crammed into the deep pockets of his
dressing-gown. The room was dense with cigarette-smoke, and his
figure appeared misty in the blue haze. I went back to bed and lay
awake for an hour. When finally I dozed off it was to the
accompaniment of those rhythmic footfalls in the library.
I rose at eight o’clock. It was a dark, dismal Sunday, and I had my
coffee in the living-room by electric light. When I glanced into the
library at nine Vance was still there, sitting at his desk. The reading-
lamp was burning, but the fire on the hearth had died out. Returning
to the living-room, I tried to interest myself in the Sunday
newspapers; but after scanning the accounts of the Greene case I lit
my pipe and drew up my chair before the grate.
It was nearly ten o’clock when Vance appeared at the door. All
night he had been up, wrestling with his self-imposed problem; and
the devitalizing effects of this long, sleepless concentration showed
on him only too plainly. There were shadowed circles round his eyes;
his mouth was drawn; and even his shoulders sagged wearily. But,
despite the shock his appearance gave me, my dominant emotion
was one of avid curiosity. I wanted to know the outcome of his all-
night vigil; and as he came into the room I gave him a look of
questioning expectancy.
When his eyes met mine he nodded slowly.
“I’ve traced the design,” he said, holding out his hands to the
warmth of the fire. “And it’s more horrible than I even imagined.” He
was silent for some minutes. “Telephone Markham for me, will you,
Van? Tell him I must see him at once. Ask him to come to breakfast.
Explain that I’m a bit fagged.”
He went out, and I heard him calling to Currie to prepare his bath.
I had no difficulty in inducing Markham to breakfast with us after I
had explained the situation; and in less than an hour he arrived.
Vance was dressed and shaved, and looked considerably fresher
than when I had first seen him that morning; but he was still pale,
and his eyes were fatigued.
No mention was made of the Greene case during breakfast, but
when we had sought easy chairs in the library, Markham could
withhold his impatience no longer.
“Van intimated over the phone that you had made something out
of the summary.”
 “Yes.” Vance spoke dispiritedly. “I’ve fitted all the items together.
And it’s damnable! No wonder the truth escaped us.”
Markham leaned forward, his face tense, unbelieving.
“You know the truth?”
“Yes, I know,” came the quiet answer. “That is, my brain has told
me conclusively who’s at the bottom of this fiendish affair; but even
now—in the daylight—I can’t credit it. Everything in me revolts
against the acceptance of the truth. The fact is, I’m almost afraid to
accept it. . . . Dash it all, I’m getting mellow. Middle-age has crept
upon me.” He attempted to smile, but failed.
Markham waited in silence.
“No, old man,” continued Vance; “I’m not going to tell you now. I
can’t tell you until I’ve looked into one or two matters. You see, the
pattern is plain enough, but the recognizable objects, set in their new
relationships, are grotesque—like the shapes in an awful dream. I
must first touch them and measure them to make sure that they’re
not, after all, mere abortive vagaries.”
“And how long will this verification take?” Markham knew there
was no use to try to force the issue. He realized that Vance was fully
conscious of the seriousness of the situation, and respected his
decision to investigate certain points before revealing his
conclusions.
“Not long, I hope.” Vance went to his desk and wrote something
on a piece of paper, which he handed to Markham. “Here’s a list of
the five books in Tobias’s library that showed signs of having been
read by the nocturnal visitor. I want those books, Markham—
immediately. But I don’t want any one to know about their being
taken away. Therefore, I’m going to ask you to phone Nurse O’Brien
to get Mrs. Greene’s key and secure them when no one is looking.
Tell her to wrap them up and give them to the detective on guard in
the house with instructions to bring them here. You can explain to
her what section of the book-shelves they’re in.”
Markham took the paper and rose without a word. At the door of
the den, however, he paused.
“Do you think it wise for the man to leave the house?”
“It won’t matter,” Vance told him. “Nothing more can happen there
at present.”
 Markham went on into the den. In a few minutes he returned.
“The books will be here in half an hour.”
When the detective arrived with the package Vance unwrapped it
and laid the volumes beside his chair.
“Now, Markham, I’m going to do some reading. You won’t mind,
what?” Despite his casual tone, it was evident that an urgent
seriousness underlay his words.
Markham got up immediately; and again I marvelled at the
complete understanding that existed between these two disparate
men.
“I have a number of personal letters to write,” he said, “so I’ll run
along. Currie’s omelet was excellent.—When shall I see you again? I
could drop round at tea-time.”
Vance held out his hand with a look bordering on affection.
“Make it five o’clock. I’ll be through with my perusings by then.
And thanks for your tolerance.” Then he added gravely: “You’ll
understand, after I’ve told you everything, why I wanted to wait a bit.”
When Markham returned that afternoon a little before five Vance
was still reading in the library; but shortly afterward he joined us in
the living-room.
“The picture clarifies,” he said. “The fantastic images are
gradually taking on the aspect of hideous realities. I’ve substantiated
several points, but a few facts still need corroboration.”
“To vindicate your hypothesis?”
“No, not that. The hypothesis is self-proving. There’s no doubt as
to the truth. But—dash it all, Markham!—I refuse to accept it until
every scrap of evidence has been incontestably sustained.”
“Is the evidence of such a nature that I can use it in a court of
law?”
“That is something I refuse even to consider. Criminal
proceedings seem utterly irrelevant in the present case. But I
suppose society must have its pound of flesh, and you—the duly
elected Shylock of God’s great common people—will no doubt wield
the knife. However, I assure you I shall not be present at the
butchery.”
Markham studied him curiously.
 “Your words sound rather ominous. But if, as you say, you have
discovered the perpetrator of these crimes, why shouldn’t society
exact punishment?”
“If society were omniscient, Markham, it would have a right to sit
in judgment. But society is ignorant and venomous, devoid of any
trace of insight or understanding. It exalts knavery, and worships
stupidity. It crucifies the intelligent, and puts the diseased in
dungeons. And, withal, it arrogates to itself the right and ability to
analyze the subtle sources of what it calls ‘crime,’ and to condemn to
death all persons whose inborn and irresistible impulses it does not
like. That’s your sweet society, Markham—a pack of wolves watering
at the mouth for victims on whom to vent its organized lust to kill and
flay.”
Markham regarded him with some astonishment and
considerable concern.
“Perhaps you are preparing to let the criminal escape in the
present case,” he said, with the irony of resentment.
“Oh, no,” Vance assured him. “I shall turn your victim over to you.
The Greene murderer is of a particularly vicious type, and should be
rendered impotent. I was merely trying to suggest that the electric
chair—that touchin’ device of your beloved society—is not quite the
correct method of dealing with this culprit.”
“You admit, however, that he is a menace to society.”
“Undoubtedly. And the hideous thing about it is that this
tournament of crime at the Greene mansion will continue unless we
can put a stop to it. That’s why I am being so careful. As the case
now stands, I doubt if you could even make an arrest.”
When tea was over Vance got up and stretched himself.
“By the by, Markham,” he said offhandedly, “have you received
any report on Sibella’s activities?”
“Nothing important. She’s still in Atlantic City, and evidently
intends to stay there for some time. She phoned Sproot yesterday to
send down another trunkful of her clothes.”
“Did she, now? That’s very gratifyin’.” Vance walked to the door
with sudden resolution. “I think I’ll run out to the Greenes’ for a little
while. I sha’n’t be gone over an hour. Wait for me here, Markham—
there’s a good fellow; I don’t want my visit to have an official flavor.
There’s a new Simplicissimus on the table to amuse you till I return.
Con it and thank your own special gods that you have no Thöny or
Gulbranssen in this country to caricature your Gladstonian features.”
As he spoke he beckoned to me, and, before Markham could
question him, we passed out into the hall and down the stairs.
Fifteen minutes later a taxicab set us down before the Greene
mansion.
Sproot opened the door for us, and Vance, with only a curt
greeting, led him into the drawing-room.
“I understand,” he said, “that Miss Sibella phoned you yesterday
from Atlantic City and asked to have a trunk shipped to her.”
Sproot bowed. “Yes, sir. I sent the trunk off last night.”
“What did Miss Sibella say to you over the phone?”
“Very little, sir—the connection was not good. She said merely
that she had no intention of returning to New York for a considerable
time and needed more clothes than she had taken with her.”
“Did she ask how things were going at the house here?”
“Only in the most casual way, sir.”
“Then she didn’t seem apprehensive about what might happen
here while she was away?”
“No, sir. In fact—if I may say so without disloyalty—her tone of
voice was quite indifferent, sir.”
“Judging from her remarks about the trunk, how long would you
say she intends to be away?”
Sproot considered the matter.
“That’s difficult to say, sir. But I would go so far as to venture the
opinion that Miss Sibella intends to remain in Atlantic City for a
month or more.”
Vance nodded with satisfaction.
“And now, Sproot,” he said, “I have a particularly important
question to ask you. When you first went into Miss Ada’s room on
the night she was shot and found her on the floor before the
dressing-table, was the window open? Think! I want a positive
answer. You know the window is just beside the dressing-table and
overlooks the steps leading to the stone balcony. Was it open or
shut?”
 Sproot contracted his brows and appeared to be recalling the
scene. Finally he spoke, and there was no doubt in his voice.
“The window was open, sir. I recall it now quite distinctly. After Mr.
Chester and I had lifted Miss Ada to the bed, I closed it at once for
fear she would catch cold.”
“How far open was the window?” asked Vance with eager
impatience.
“Eight or nine inches, sir, I should say. Perhaps a foot.”
“Thank you, Sproot. That will be all. Now please tell the cook I
want to see her.”
Mrs. Mannheim came in a few minutes later, and Vance indicated
a chair near the desk-light. When the woman had seated herself he
stood before her and fixed her with a stern, implacable gaze.
“Frau Mannheim, the time for truth-telling has come. I am here to
ask you a few questions, and unless I receive a straight answer to
them I shall report you to the police. You will, I assure you, receive
no consideration at their hands.”
The woman tightened her lips stubbornly and shifted her eyes,
unable to meet Vance’s penetrating stare.
“You told me once that your husband died in New Orleans
thirteen years ago. Is that correct?”
Vance’s question seemed to relieve her mind, and she answered
readily.
“Yes, yes. Thirteen years ago.”
“What month?”
“In October.”
“Had he been ill long?”
“About a year.”
“What was the nature of his illness?”
Now a look of fright came into her eyes.
“I—don’t know—exactly,” she stammered. “The doctors didn’t let
me see him.”
“He was in a hospital?”
She nodded several times rapidly. “Yes—a hospital.”
“And I believe you told me, Frau Mannheim, that you saw Mr.
Tobias Greene a year before your husband’s death. That would have
been about the time your husband entered the hospital—fourteen
years ago.”
She looked vaguely at Vance, but made no reply.
“And it was exactly fourteen years ago that Mr. Greene adopted
Ada.”
The woman caught her breath sharply. A look of panic contorted
her face.
“So when your husband died,” continued Vance, “you came to Mr.
Greene, knowing he would give you a position.”
He went up to her and touched her filially on the shoulder.
“I have suspected for some time, Frau Mannheim,” he said kindly,
“that Ada is your daughter. It’s true, isn’t it?”
With a convulsive sob the woman hid her face in her apron.
“I gave Mr. Greene my word,” she confessed brokenly, “that I
wouldn’t tell any one—not even Ada—if he let me stay here—to be
near her.”
“You haven’t told any one,” Vance consoled her. “It was not your
fault that I guessed it. But why didn’t Ada recognize you?”
“She had been away—to school—since she was five.”
When Mrs. Mannheim left us a little later Vance had succeeded in
allaying her apprehension and distress. He then sent for Ada.
As she entered the drawing-room the troubled look in her eyes
and the pallor of her cheeks told clearly of the strain she was under.
Her first question voiced the fear uppermost in her mind.
“Have you found out anything, Mr. Vance?” She spoke with an air
of pitiful discouragement. “It’s terrible alone here in this big house—
especially at night. Every sound I hear . . .”
“You mustn’t let your imagination get the better of you, Ada,”
Vance counselled her. Then he added: “We know a lot more now
than we did, and before long, I hope, all your fears will be done away
with. In fact, it’s in regard to what we’ve found out that I’ve come
here to-day. I thought perhaps you could help me again.”
“If only I could! But I’ve thought and thought. . . .”
Vance smiled.
“Let us do the thinking, Ada.—What I wanted to ask you is this:
do you know if Sibella speaks German well?”
The girl appeared surprised.
 “Why, yes. And so did Julia and Chester and Rex. Father insisted
on their learning it. And he spoke it too—almost as well as he spoke
English. As for Sibella, I’ve often heard her and Doctor Von talking in
German.”
“But she spoke with an accent, I suppose.”
“A slight accent—she’d never been long in Germany. But she
spoke very well German.”
“That’s what I wanted to be sure of.”
“Then you do know something!” Her voice quavered with
eagerness. “Oh, how long before this awful suspense will be over?
Every night for weeks I’ve been afraid to turn out my lights and go to
sleep.”
“You needn’t be afraid to turn out your lights now,” Vance assured
her. “There won’t be any more attempts on your life, Ada.”
She looked at him for a moment searchingly, and something in
his manner seemed to hearten her. When we took our leave the
color had come back to her cheeks.
Markham was pacing the library restlessly when we arrived
home.
“I’ve checked several more points,” Vance announced. “But I’ve
missed the important one—the one that would explain the
unbelievable hideousness of the thing I’ve unearthed.”
He went directly into the den, and we could hear him telephoning.
Returning a few minutes later, he looked anxiously at his watch.
Then he rang for Currie and ordered his bag packed for a week’s
trip.
“I’m going away, Markham,” he said. “I’m going to travel—they
say it broadens the mind. My train departs in less than an hour; and
I’ll be away a week. Can you bear to be without me for so long?
However, nothing will happen in connection with the Greene case
during my absence. In fact, I’d advise you to shelve it temporarily.”
He would say no more, and in half an hour he was ready to go.
“There’s one thing you can do for me while I’m away,” he told
Markham, as he slipped into his overcoat. “Please have drawn up for
me a complete and detailed weather report from the day preceding
Julia’s death to the day following Rex’s murder.”
 He would not let either Markham or me accompany him to the
station, and we were left in ignorance of even the direction in which
his mysterious trip was to take him.
 CHAPTER XXV.
The Capture
(Monday, December 13; 4 p. m.)

It was eight days before Vance returned to New York. He arrived

on the afternoon of Monday, December 13, and, after he had had his
tub and changed his clothes, he telephoned Markham to expect him
in half an hour. He then ordered his Hispano-Suiza from the garage;
and by this sign I knew he was under a nervous strain. In fact, he
had spoken scarcely a dozen words to me since his return, and as
he picked his way down-town through the late afternoon traffic he
was gloomy and preoccupied. Once I ventured to ask him if his trip
had been successful, and he had merely nodded. But when we
turned into Centre Street he relented a little, and said:
“There was never any doubt as to the success of my trip, Van. I
knew what I’d find. But I didn’t dare trust my reason; I had to see the
records with my own eyes before I’d capitulate unreservedly to the
conclusion I’d formed.”
Both Markham and Heath were waiting for us in the District
Attorney’s office. It was just four o’clock, and the sun had already
dropped below the New York Life Building which towered above the
old Criminal Courts structure a block to the southwest.
“I took it for granted you had something important to tell me,” said
Markham; “so I asked the Sergeant to come here.”
“Yes, I’ve much to tell.” Vance had thrown himself into a chair,
and was lighting a cigarette. “But first I want to know if anything has
happened in my absence.”
“Nothing. Your prognostication was quite accurate. Things have
been quiet and apparently normal at the Greene mansion.”