Visit https://ebookultra.

com to download the full version and

explore more ebooks or textbooks

Numerical issues in statistical computing for

social scientist 1st Edition Micah Altman

_____ Click the link below to download _____

https://ebookultra.com/download/numerical-issues-in-
statistical-computing-for-social-scientist-1st-edition-
micah-altman/

Explore and download more ebooks or textbooks at ebookultra.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Numerical and Statistical Methods for Bioengineering

Applications in MATLAB 1st Edition Michael R. King

https://ebookultra.com/download/numerical-and-statistical-methods-for-
bioengineering-applications-in-matlab-1st-edition-michael-r-king/

Learning SciPy for Numerical and Scientific Computing

Second Edition Sergio J. Rojas G.

https://ebookultra.com/download/learning-scipy-for-numerical-and-
scientific-computing-second-edition-sergio-j-rojas-g/

Numerical Mathematics and Computing 7th Edition E. Ward

Cheney

https://ebookultra.com/download/numerical-mathematics-and-
computing-7th-edition-e-ward-cheney/

Numerical Methods of Scientific Computing 2nd Edition J.

Van Kan

https://ebookultra.com/download/numerical-methods-of-scientific-
computing-2nd-edition-j-van-kan/
Numerical Computing with MATLAB Revised Reprint 2nd
Edition Cleve B. Moler

https://ebookultra.com/download/numerical-computing-with-matlab-
revised-reprint-2nd-edition-cleve-b-moler/

Ns Network Simulator for Beginners 1st Edition Eitan

Altman

https://ebookultra.com/download/ns-network-simulator-for-
beginners-1st-edition-eitan-altman/

Using Statistical Methods for Water Quality Management

Issues Problems and Solutions 1st Edition Graham B.
Mcbride
https://ebookultra.com/download/using-statistical-methods-for-water-
quality-management-issues-problems-and-solutions-1st-edition-graham-b-
mcbride/

Mexico Economic Political and Social Issues Economic

Political and Social Issues 1st Edition Maria L. Castro

https://ebookultra.com/download/mexico-economic-political-and-social-
issues-economic-political-and-social-issues-1st-edition-maria-l-
castro/

Micah 1st Edition J. C. De Moor

https://ebookultra.com/download/micah-1st-edition-j-c-de-moor/
Numerical issues in statistical computing for social
scientist 1st Edition Micah Altman Digital Instant
Download
Author(s): Micah Altman, Jeff Gill, Michael P. McDonald
ISBN(s): 9780471236337, 0471236330
Edition: 1
File Details: PDF, 4.76 MB
Year: 2003
Language: english
Numerical Issues in
Statistical Computing
for the Social Scientist

MICAH ALTMAN
JEFF GILL
MICHAEL P. McDONALD

A JOHN WILEY & SONS, INC., PUBLICATION

Numerical Issues in
Statistical Computing
for the Social Scientist
ii

WILEY SERIES IN PROBABILITY AND STATISTICS

Established by WALTER A. SHEWHART and SAMUEL S. WILKS

Editors: David J. Balding, Noel A. C. Cressie, Nicholas I. Fisher,

Iain M. Johnstone, J. B. Kadane, Louise M. Ryan, David W. Scott,
Adrian F. M. Smith, Jozef L. Teugels;
Editors Emeriti: Vic Barnett, J. Stuart Hunter, David G. Kendall

A complete list of the titles in this series appears at the end of this volume.
Numerical Issues in
Statistical Computing
for the Social Scientist

MICAH ALTMAN
JEFF GILL
MICHAEL P. McDONALD

A JOHN WILEY & SONS, INC., PUBLICATION

Copyright

c 2004 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any
form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise,
except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without
either the prior written permission of the Publisher, or authorization through payment of the
appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers,
MA 01923, 978-750-8400, fax 978-750-4470, or on the web at www.copyright.com. Requests to
the Publisher for permission should be addressed to the Permissions Department, John Wiley &
Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, e-mail:
permreq@wiley.com.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best
efforts in preparing this book, they make no representations or warranties with respect to the
accuracy or completeness of the contents of this book and specifically disclaim any implied
warranties of merchantability or fitness for a particular purpose. No warranty may be created or
extended by sales representatives or written sales materials. The advice and strategies contained
herein may not be suitable for your situation. You should consult with a professional where
appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other
commercial damages, including but not limited to special, incidental, consequential, or other
damages.

For general information on our other products and services please contact our Customer Care
Department within the U.S. at 877-762-2974, outside the U.S. at 317-572-3993 or
fax 317-572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in
print, however, may not be available in electronic format.

Library of Congress Cataloging-in-Publication Data:

Altman, Micah.
Numerical issues in statistical computing for the social scientist / Micah Altman, Jeff
Gill, Michael P. McDonald.
p. cm.—(Wiley series in probability and statistics)
Includes bibliographical references and index.
ISBN 0-471-23633-0 (acid-free paper)
1. Statistics–Data processing. 2. Social sciences–Statistical methods–Data processing.
I. Gill, Jeff. II. McDonald, Michael P., 1967–III. Title. IV. Series.

QA276.4.A398 2004
519.5–dc21 2003053470

Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1
 Contents

Preface xi

1 Introduction: Consequences of Numerical Inaccuracy 1

1.1 Importance of Understanding Computational Statistics 1

1.2 Brief History: Duhem to the Twenty-First Century 3
1.3 Motivating Example: Rare Events Counts Models 6
1.4 Preview of Findings 10

2 Sources of Inaccuracy in Statistical Computation 12

2.1 Introduction 12
2.1.1 Revealing Example: Computing the Coefficient
Standard Deviation 12
2.1.2 Some Preliminary Conclusions 13
2.2 Fundamental Theoretical Concepts 15
2.2.1 Accuracy and Precision 15
2.2.2 Problems, Algorithms, and Implementations 15
2.3 Accuracy and Correct Inference 18
2.3.1 Brief Digression: Why Statistical Inference Is Harder
in Practice Than It Appears 20
2.4 Sources of Implementation Errors 21
2.4.1 Bugs, Errors, and Annoyances 22
2.4.2 Computer Arithmetic 23
2.5 Algorithmic Limitations 29
2.5.1 Randomized Algorithms 30
2.5.2 Approximation Algorithms for Statistical Functions 31
2.5.3 Heuristic Algorithms for Random Number Generation 32
2.5.4 Local Search Algorithms 39
2.6 Summary 41

v
vi CONTENTS

3 Evaluating Statistical Software 44

3.1 Introduction 44
3.1.1 Strategies for Evaluating Accuracy 44
3.1.2 Conditioning 47
3.2 Benchmarks for Statistical Packages 48
3.2.1 NIST Statistical Reference Datasets 49
3.2.2 Benchmarking Nonlinear Problems with StRD 51
3.2.3 Analyzing StRD Test Results 53
3.2.4 Empirical Tests of Pseudo-Random Number Generation 54
3.2.5 Tests of Distribution Functions 58
3.2.6 Testing the Accuracy of Data Input and Output 60
3.3 General Features Supporting Accurate and Reproducible Results 63
3.4 Comparison of Some Popular Statistical Packages 64
3.5 Reproduction of Research 65
3.6 Choosing a Statistical Package 69

4 Robust Inference 71
4.1 Introduction 71
4.2 Some Clarification of Terminology 71
4.3 Sensitivity Tests 73
4.3.1 Sensitivity to Alternative Implementations
and Algorithms 73
4.3.2 Perturbation Tests 75
4.3.3 Tests of Global Optimality 84
4.4 Obtaining More Accurate Results 91
4.4.1 High-Precision Mathematical Libraries 92
4.4.2 Increasing the Precision of Intermediate Calculations 93
4.4.3 Selecting Optimization Methods 95
4.5 Inference for Computationally Difficult Problems 103
4.5.1 Obtaining Confidence Intervals
with Ill-Behaved Functions 104
4.5.2 Interpreting Results in the Presence
of Multiple Modes 106
4.5.3 Inference in the Presence of Instability 114

5 Numerical Issues in Markov Chain Monte Carlo Estimation 118

5.1 Introduction 118
5.2 Background and History 119
5.3 Essential Markov Chain Theory 120
CONTENTS vii

5.3.1 Measure and Probability Preliminaries 120

5.3.2 Markov Chain Properties 121
5.3.3 The Final Word (Sort of) 125
5.4 Mechanics of Common MCMC Algorithms 126
5.4.1 Metropolis–Hastings Algorithm 126
5.4.2 Hit-and-Run Algorithm 127
5.4.3 Gibbs Sampler 128
5.5 Role of Random Number Generation 129
5.5.1 Periodicity of Generators and MCMC Effects 130
5.5.2 Periodicity and Convergence 132
5.5.3 Example: The Slice Sampler 135
5.5.4 Evaluating WinBUGS 137
5.6 Absorbing State Problem 139
5.7 Regular Monte Carlo Simulation 140
5.8 So What Can Be Done? 141

6 Numerical Issues Involved in Inverting Hessian Matrices 143

Jeff Gill and Gary King
6.1 Introduction 143
6.2 Means versus Modes 145
6.3 Developing a Solution Using Bayesian Simulation Tools 147
6.4 What Is It That Bayesians Do? 148
6.5 Problem in Detail: Noninvertible Hessians 149
6.6 Generalized Inverse/Generalized Cholesky Solution 151
6.7 Generalized Inverse 151
6.7.1 Numerical Examples of the Generalized Inverse 154
6.8 Generalized Cholesky Decomposition 155
6.8.1 Standard Algorithm 156
6.8.2 Gill–Murray Cholesky Factorization 156
6.8.3 Schnabel–Eskow Cholesky Factorization 158
6.8.4 Numerical Examples of the Generalized
Cholesky Decomposition 158
6.9 Importance Sampling and Sampling Importance Resampling 160
6.9.1 Algorithm Details 160
6.9.2 SIR Output 162
6.9.3 Relevance to the Generalized Process 163
6.10 Public Policy Analysis Example 163
6.10.1 Texas 164
6.10.2 Florida 168
CONTENTS ix

9 Spatial Regression Models 219

James P. LeSage
9.1 Introduction 219
9.2 Sample Data Associated with Map Locations 219
9.2.1 Spatial Dependence 219
9.2.2 Specifying Dependence Using Weight Matrices 220
9.2.3 Estimation Consequences of Spatial Dependence 222
9.3 Maximum Likelihood Estimation of Spatial Models 223
9.3.1 Sparse Matrix Algorithms 224
9.3.2 Vectorization of the Optimization Problem 225
9.3.3 Trade-offs between Speed and Numerical Accuracy 226
9.3.4 Applied Illustrations 228
9.4 Bayesian Spatial Regression Models 229
9.4.1 Bayesian Heteroscedastic Spatial Models 230
9.4.2 Estimation of Bayesian Spatial Models 231
9.4.3 Conditional Distributions for the SAR Model 232
9.4.4 MCMC Sampler 234
9.4.5 Illustration of the Bayesian Model 234
9.5 Conclusions 236

10 Convergence Problems in Logistic Regression 238

Paul Allison
10.1 Introduction 238
10.2 Overview of Logistic Maximum Likelihood Estimation 238
10.3 What Can Go Wrong? 240
10.4 Behavior of the Newton–Raphson Algorithm under Separation 243
10.4.1 Specific Implementations 244
10.4.2 Warning Messages 244
10.4.3 False Convergence 246
10.4.4 Reporting of Parameter Estimates and Standard Errors 247
10.4.5 Likelihood Ratio Statistics 247
10.5 Diagnosis of Separation Problems 247
10.6 Solutions for Quasi-Complete Separation 248
10.6.1 Deletion of Problem Variables 248
10.6.2 Combining Categories 248
10.6.3 Do Nothing and Report Likelihood Ratio
Chi-Squares 249
10.6.4 Exact Inference 249
x CONTENTS

10.6.5 Bayesian Estimation 250

10.6.6 Penalized Maximum Likelihood Estimation 250
10.7 Solutions for Complete Separation 251
10.8 Extensions 252

11 Recommendations for Replication and Accurate Analysis 253

11.1 General Recommendations for Replication 253
11.1.1 Reproduction, Replication, and Verification 254
11.1.2 Recreating Data 255
11.1.3 Inputting Data 256
11.1.4 Analyzing Data 257
11.2 Recommendations for Producing Verifiable Results 259
11.3 General Recommendations for Improving the Numeric
Accuracy of Analysis 260
11.4 Recommendations for Particular Statistical Models 261
11.4.1 Nonlinear Least Squares and Maximum Likelihood 261
11.4.2 Robust Hessian Inversion 262
11.4.3 MCMC Estimation 263
11.4.4 Logistic Regression 265
11.4.5 Spatial Regression 266
11.5 Where Do We Go from Here? 266

Bibliography 267

Author Index 303

Subject Index 315

 Preface

Overview
This book is intended to serve multiple purposes. In one sense it is a pure research
book in the traditional manner: new principles, new algorithms, and new solu-
tions. But perhaps more generally it is a guidebook like those used by naturalists
to identify wild species. Our “species” are various methods of estimation requir-
ing advanced statistical computing: maximum likelihood, Markov chain Monte
Carlo, ecological inference, nonparametrics, and so on. Only a few are wild; most
are reasonably domesticated.
A great many empirical researchers in the social sciences take computational
factors for granted: “For the social scientist, software is a tool, not an end in
itself” (MacKie-Mason 1992). Although an extensive literature exists on statis-
tical computing in statistics, applied mathematics, and embedded within various
natural science fields, there is currently no such guide tailored to the needs of
the social sciences. Although an abundance of package-specific literature and a
small amount of work at the basic, introductory level exists, a text is lacking
that provides social scientists with modern tools, tricks, and advice, yet remains
accessible through explanation and example.
The overall purpose of this work is to address what we see as a serious defi-
ciency in statistical work in the social and behavioral sciences, broadly defined.
Quantitative researchers in these fields rely on statistical and mathematical com-
putation as much as any of their colleagues in the natural sciences, yet there is
less appreciation for the problems and issues in numerical computation. This book
seeks to rectify this discrepancy by providing a rich set of interrelated chapters on
important aspects of social science statistical computing that will guide empirical
social scientists past the traps and mines of modern statistical computing.
The lack of a bridging work between standard statistical texts, which, at most,
touch on numerical computing issues, and the comprehensive work in statisti-
cal computing has hindered research in a number of social science fields. There
are two pathologies that can result. In one instance, the statistical computing
process fails and the user gives up and finds less sophisticated means of answer-
ing research questions. Alternatively, something disastrous happens during the
numerical calculations, yet seemingly reasonable output results. This is much
worse, because there are no indications that something has failed, and incorrect
statistical output becomes a component of the larger project.

xi
xii PREFACE

Fortunately, many of the most common problems are easy to describe and
easier still to avoid. We focus here to a great extent on problems that can occur
in maximum likelihood estimation and nonlinear regression because these are,
with the exception of simple linear models, the methods most widely used by
social scientists. False convergence, numerical instability, and problematic like-
lihood surfaces can be diagnosed without much agony by most interested social
scientists if they have specific advice about how to do so. Straightforward com-
putational techniques such as data rescaling, changes of starting values, function
reparameterization, and proper use of analytic derivatives can then be used to
reduce or eliminate many numerical problems. Other important and recent sta-
tistical approaches that we discuss are ecological inference, logistic regression,
Markov chain Monte Carlo, and spatial analysis.

Starters
In this book we introduce the basic principles of numerical computation, outlines
the optimization process, and provides specific tools to assess the sensitivity of
the subsequent results to problems with these data or model. The reader is not
required to have an extensive background in mathematical statistics, advanced
matrix algebra, or computer science. In general, the reader should have at least
a year of statistics training, including maximum likelihood estimation, modest
matrix algebra, and some basic calculus. In addition, rudimentary programming
knowledge in a statistical package or compiled language is required to understand
and implement the ideas herein.
Some excellent sources for addressing these preliminaries can be found in the
following sources.

• Introductory statistics. A basic introductory statistics course, along the lines

of such texts as: Moore and McCabe’s Introduction to the Practice of Statis-
tics (2002), Moore’s The Basic Practice of Statistics (1999), Basic Statistics
for the Social and Behavioral Sciences by Diekhoff (1996), Blalock’s well-
worn Social Statistics (1979), Freedman et al.’s Statistics (1997), Ame-
miya’s Introduction to Statistics and Econometrics (1994), or Statistics for
the Social Sciences by Sirkin (1999).
• Elementary matrix algebra. Some knowledge of matrix algebra, roughly at
the level of Greene’s (2003) introductory appendix, or the first half of the
undergraduate texts by either Axler (1997) or Anton and Rorres (2000).
It will not be necessary for readers to have an extensive knowledge of
linear algebra or experience with detailed calculations. Instead, knowledge
of the structure of matrices, matrix and vector manipulation, and essential
symbology will be assumed. Having said that, two wonderful reference
books that we advise owning are the theory book by Lax (1997), and the
aptly entitled book by Harville (1997), Matrix Algebra from a Statisticians
Perspective.
PREFACE xiii

• Basic calculus. Elementary knowledge of calculus is important. Helpful,

basic, and inexpensive basic texts include Kleppner and Ramsey (1985),
Bleau (1994), Thompson and Gardner (1998), and for a very basic intro-
duction, see Downing (1996). Although we avoid extensive derivations, this
material is occasionally helpful.

Programming
Although knowledge of programming is not required, most readers of this book
are, or should be, programmers. We do not mean necessarily in the sense of
generating hundreds of lines of FORTRAN code between seminars. By program-
ming we mean working with statistical languages: writing likelihood functions
in Gauss, R, or perhaps even Stata, coding solutions in WinBUGS, or manip-
ulating procedures in SAS. If all available social science statistical solutions were
available as point-and-click solutions in SPSS, there would not be very many
truly interesting models in print.
There are two, essentially diametric views on programming among academic
practitioners in the social sciences. One is emblemized by a well-known quote
from Hoare (1969, p. 576): “Computer programming is an exact science in that
all the properties of a program and all the consequences of executing it in any
given environment can, in principle, be found out from the text of the program
itself by means of purely deductive reasoning.” A second is by Knuth (1973): “It
can be an aesthetic experience much like composing poetry or music.” Our per-
spective on programming agrees with both experts; programming is a rigorous
and exacting process, but it should also be creative and fun. It is a reward-
ing activity because practitioners can almost instantly see the fruits of their
labor. We give extensive guidance here about the practice of statistical pro-
gramming because it is important for doing advanced work and for generating
high-quality work.

Layout of Book and Course Organization

There are two basic sections to this book. The first comprises four chapters and
focuses on general issues and concerns in statistical computing. The goal in this
section is to review important aspects of numerical maximum likelihood and related
estimation procedures while identifying specific problems. The second section is
a series of six chapters outlining specific problems that center on problems that
originate in different disciplines but are not necessarily contained within. Given
the extensive methodological cross-fertilization that occurs in the social sciences,
these chapters should have more than a narrow appeal. The last chapter provides a
summary of recommendations from previous chapters and an extended discussion
of methods for ensuring the general replicability of one’s research.
The book is organized as a single-semester assignment accompanying text.
Obviously, this means that some topics are treated with less detail than in a
xiv PREFACE

fully developed mathematical statistics text that would be assigned in a one-year

statistics department course. However, there is a sufficient set of references to
lead interested readers into more detailed works.
A general format is followed within each chapter in this work, despite widely
varying topics. A specific motivation is given for the material, followed by a
detailed exposition of the tool (mode finding, EI, logit estimation, MCMC, etc.).
The main numerical estimation issues are outlined along with various means
of avoiding specific common problems. Each point is illustrated using data that
social scientists care about and can relate to. This last point is not trivial; a
great many books in overlapping areas focus on examples from biostatistics, and
the result is often to reduce reader interest and perceived applicability in the
social sciences. Therefore, every example is taken from the social and behavioral
sciences, including: economics, marketing, psychology, public policy, sociology,
political science, and anthropology.
Many researchers in quantitative social science will simply read this book
from beginning to end. Researchers who are already familiar with the basics
of statistical computation may wish to skim the first several chapters and pay
particular attention to Chapters 4, 5, 6, and 11, as well as chapters specific to the
methods being investigated.
Because of the diversity of topics and difficulty levels, we have taken pains to
ensure that large sections of the book are approachable by other audiences. For
those who do not have the time or training to read the entire book, we recommend
the following:

• Undergraduates in courses on statistics or research methodology, will find

a gentle introduction to statistical computation and its importance in Sec-
tion 1.1 and Chapter 2. These may be read without prerequisites.
• Graduate students doing any type of quantitative research will wish to
read the introductory chapters as well, and will find Chapters 3 and 11
useful and approachable. Graduate students using more advanced statistical
models should also read Chapters 5 and 8, although these require more
some mathematical background.
• Practitioners may prefer to skip the introduction, and start with Chapters 3,
4, and 11, as well as other chapters specific to the methods they are using
(e.g., nonlinear models, MCMC, ecological inference, spatial methods).

However, we hope readers will enjoy the entire work. This is intended to be a
research work as well as a reference work, so presumably experienced researchers
in this area will still find some interesting new points and views within.

Web Site
Accompanying this book is a Web site: <http://www.hmdc.harvard.edu/
numerical issues/>. This site contains links to many relevant resources,
PREFACE xv

including bibliographies, discussion-lists, benchmark data, high-precision libraries,

and optimization software.
In addition, the Web site includes links to all of the code and data used in this
book and not otherwise described in detail, in order to assist other scholars in
carrying out similar analyses on other datasets.

Debts and Dedications

We would like to thank the support of our host institutions: Harvard University,
University of Florida, and George Mason University. All three of us have worked
in and enjoyed the Harvard–MIT Data Center as a research area, as a provider of
data, and as an intellectual environment to test these ideas. We also unjammed
the printer a lot and debugged the e-mail system on occasion while there. We
thank Gary King for supporting our presence at the center.
The list of people to thank in this effort is vast. We would certainly be
remiss without mentioning Chris Achen, Bob Anderson, Attic Access, Neal Beck,
Janet Box–Steffensmeier, Barry Burden, Dieter Burrell, George Casella, Suzie De
Boef, Scott Desposato, Karen Ferree, John Fox, Charles Franklin, Hank Heitowit,
Michael Herron, Jim Hobert, James Honaker, Simon Jackman, Bill Jacoby, David
James, Dean Lacey, Andrew Martin, Michael Martinez, Rogerio Mattos, Ken
McCue, Ken Meier, Kylie Mills, Chris Mooney, Jonathan Nagler, Kevin Quinn,
Ken Shotts, Kevin Smith, Wendy Tam Cho, Alvaro Veiga, William Wei, Guy
Whitten, Jason Wittenberg, Dan Wood, and Chris Zorn (prison rodeo consultant
to the project). A special thanks go to our contributing authors, Paul Allison,
Gary King, James LeSage, and Bruce McCullough, for their excellent work, tire-
less rewriting efforts, and general patience with the three of us. Special thanks
also go to our editor, Steve Quigley, as well, since this project would not exist
without his inspiration, prodding, and general guidance.
Significant portions of this book, especially Chapters 2, 3, and 11, are based
in part upon research supported by National Science Foundation Award No.
11S-987 47 47.
This project was typeset using LATEX and associated tools from the TEX world
on a Linux cluster housed at the Harvard–MIT Data Center. We used the John
Wiley & Sons LATEX style file with the default computer modern font. All of this
produced very nice layouts with only moderate effort on our part.
CHAPTER 1

Introduction: Consequences
of Numerical Inaccuracy

1.1 IMPORTANCE OF UNDERSTANDING

COMPUTATIONAL STATISTICS

How much pollution is bad for you? Well-known research conducted from 1987 to
1994 linked small-particle air pollution to health problems in 90 U.S. cities. These
findings were considered reliable and were influential in shaping public policy.
Recently, when the same scientists attempted to replicate their own findings,
they produced different results with the same data—results that showed a much
weaker link between air pollution and health problems. “[The researchers] re-
examined the original figures and found that the problem lay with how they used
off-the-shelf statistical software to identify telltale patterns that are somewhat
akin to ripples from a particular rock tossed into a wavy sea. Instead of adjusting
the program to the circumstances that they were studying, they used standard
default settings for some calculations. That move apparently introduced a bias in
the results, the team says in the papers on the Web” (Revkin, 2002).
Problems with numerical applications are practically as old as computers: In
1962, the Mariner I spacecraft, intended as the first probe to visit another planet,
was destroyed as a result of the incorrect coding of a mathematical formula (Neu-
mann 1995), and five years later, Longley (1967) reported on pervasive errors in the
accuracy of statistical programs’ implementation of linear regression. Unreliable
software is sometimes even expected and tolerated by experienced researchers.
Consider this report on the investigation of a high-profile incident of academic
fraud, involving the falsification of data purporting to support the discovery of
the world’s heaviest element at Lawrence Berkeley lab: “The initial suspect was
the analysis software, nicknamed Goosy, a somewhat temperamental computer
program known on occasion to randomly corrupt data. Over the years, users had
developed tricks for dealing with Goosy’s irregularities, as one might correct a
wobbling image on a TV set by slapping the side of the cabinet” (Johnson 2002).
In recent years, many of the most widely publicized examples of scientific
application failures related to software have been in the fields of space exploration

Numerical Issues in Statistical Computing for the Social Scientist, by Micah Altman, Jeff Gill,
and Michael P. McDonald
ISBN 0-471-23633-0 Copyright
c 2004 John Wiley & Sons, Inc.

1
2 INTRODUCTION: CONSEQUENCES OF NUMERICAL INACCURACY

and rocket technology. Rounding errors in numerical calculations were blamed

for the failure of the Patriot missile defense to protect an army barracks in
Dhahran from a Scud missile attack in 1991 during Operation Desert Storm
(Higham 2002). The next year, the space shuttle had difficulties in an attempted
rendezvous with Intelsat 6 because of a round-off error in the routines that
the shuttle computers used to compute distance (Neumann 1995). In 1999, two
Mars-bound spacecraft were lost, due (at least in part) to software errors—one
involving failure to check the units as navigational inputs (Carreau 2000). Numer-
ical software bugs have even affected our understanding of the basic structure
of the universe: highly publicized findings suggesting the existence of unknown
forms of matter in the universe, in violation of the “standard model,” were later
traced to numerical errors, such as failure to treat properly the sign of certain
calculations (Glanz 2002; Hayakawa and Kinoshita 2001).
The other sciences, and the social sciences in particular, have had their share of
less publicized numerical problems: Krug et al. (1988) retracted a study analyz-
ing suicide rates following natural disasters that was originally published in the
Journal of the American Medical Association, one of the world’s most prestigious
medical journals, because their software erroneously counted some deaths twice,
undermining their conclusions (see Powell et al. 1999). Leimer and Lesnoy
(1982) trace Feldstein’s (1974) erroneous conclusion that the introduction of
Social Security reduced personal savings by 50% to the existence of a sim-
ple software bug. Dewald et al. (1986), in replicating noted empirical results
appearing in the Journal of Money, Credit and Banking, discovered a number of
serious bugs in the original authors’ analyses programs. Our research and that of
others has exposed errors in articles recently published in political and social sci-
ence journals that can be traced to numerical inaccuracies in statistical software
(Altman and McDonald 2003; McCullough and Vinod 2003; Stokes 2003).
Unfortunately, numerical errors in published social science analyses can be
revealed only through replication of the research. Given the difficulty and rarity
of replication in the social sciences (Dewald et al. 1986; Feigenbaum and Levy
1993), the numerical problems reported earlier are probably the tip of the iceberg.
One is forced to wonder how much of the critical and foundational findings in a
number of fields are actually based on suspect statistical computing.
There are two primary sources of potential error in numerical algorithms pro-
grammed on computers: that numbers cannot be perfectly represented within the
limited binary world of computers, and that some algorithms are not guaranteed
to produce the desired solution.
First, small computational inaccuracies occur at the precision level of all sta-
tistical software when digits beyond the storage capacity of the computer must be
rounded or truncated. Researchers may be tempted to dismiss this threat to valid-
ity because measurement error (miscoding of data, survey sampling error, etc.)
is almost certainly an order of magnitude greater for most social science appli-
cations. But these small errors may propagate and magnify in unexpected ways
in the many calculations underpinning statistical algorithms, producing wildly
erroneous results on their own, or exacerbating the effects of measurement error.
BRIEF HISTORY: DUHEM TO THE TWENTY-FIRST CENTURY 3

Second, computational procedures may be subtly biased in ways that are hard
to detect and are sometimes not guaranteed to produce a correct solution. Ran-
dom number generators may be subtly biased: random numbers are generated by
computers through non-random, deterministic processes that mimic a sequence
of random numbers but are not genuinely random. Optimization algorithms, such
as maximum likelihood estimation, are not guaranteed to find the solution in the
presence of multiple local optima: Optimization algorithms are notably suscepti-
ble to numeric inaccuracies, and resulting coefficients may be far from their true
values, posing a serious threat to the internal validity of hypothesized relation-
ships linking concepts in the theoretical model.
An understanding of the limits of statistical software can help researchers
avoid estimation errors. For typical estimation, such as ordinary least squares
regression, well-designed off-the-shelf statistical software will generally produce
reliable estimates. For complex algorithms, our knowledge of model building
has outpaced our knowledge of computational statistics. We hope that researchers
contemplating complex models will find this book a valuable tool to aid in making
robust inference within the limits of computational statistics.
Awareness of the limits of computational statistics may further aid in model
testing. Social scientists are sometimes faced with iterative models that fail to
converge, software that produces nonsensical results, Hessians that cannot be
inverted, and other problems associated with estimation. Normally, this would
cause researchers to abandon the model or embark on the often difficult and
expensive process of gathering more data. An understanding of computational
issues can offer a more immediately available solution—such as use of more accu-
rate computations, changing algorithmic parameters of the software, or appropri-
ate rescaling of the data.

1.2 BRIEF HISTORY: DUHEM TO THE TWENTY-FIRST CENTURY

The reliability of scientific inference depends on one’s tools. As early as 1906,

French physicist and philosopher of science Pierre Duhem noted that every
scientific inference is conditioned implicitly on a constellation of background
hypotheses, including that the instruments are functioning correctly (Duhem 1991,
Sec. IV.2). The foremost of the instruments used by modern applied statisticians
is the computer.
In the early part of the twentieth century the definition of a computer to statis-
ticians was quite different from what it is today. In antiquated statistics journals
one can read where authors surprisingly mention “handing the problem over to
my computer.” Given the current vernacular, it is easy to miss what is going on
here. Statisticians at the time employed as “computers” people who specialized
in performing repetitive arithmetic. Many articles published in leading statistics
journals of the time addressed methods by which these calculations could be
made less drudgingly repetitious because it was noticed that as tedium increases
linearly, careless mistakes increase exponentially (or thereabouts). Another rather
4 INTRODUCTION: CONSEQUENCES OF NUMERICAL INACCURACY

prescient development of the time given our purpose here was the attention paid
to creating self-checking procedures where “the computer” would at regular inter-
vals have a clever means to check calculations against some summary value as
a way of detecting errors (cf. Kelley and McNemar 1929). One of the reasons
that Fisher’s normal tables (and therefore the artificial 0.01 and 0.05 significance
thresholds) were used so widely was that the task of manually calculating normal
integrals was time consuming and tedious. Computation, it turns out, played an
important role in scholarship even before the task was handed over to machines.
In 1943, Hotelling and others called attention to the accumulation of errors
in the solutions for inverting matrices in the method of least squares (Hotelling
1943) and other matrix manipulation (Turing 1948). Soon after development of
the mainframe computer, programmed regression algorithms were criticized for
dramatic inaccuracies (Longley 1967). Inevitably, we improve our software, and
just as inevitably we make our statistical methods more ambitious. Approximately
every 10 years thereafter, each new generation of statistical software has been
similarly faulted (e.g., Wampler 1980; Simon and LeSage 1988).
One of the most important statistical developments of the twentieth century
was the advent of simulation on computers. While the first simulations were
done manually by Buffon, Gosset, and others, it was not until the development
of machine-repeated calculations and electronic storage that simulation became
prevalent. In their pioneering postwar work, von Neumann and Ulam termed
this sort of work Monte Carlo simulation, presumably because it reminded them
of long-run observed odds that determine casino income (Metropolis and Ulam
1949; Von Neumann 1951). The work was conducted with some urgency in the
1950s because of the military advantage of simulating nuclear weapon designs.
One of the primary calculations performed by von Neumann and his colleagues
was a complex set of equations related to the speed of radiation diffusion of fissile
materials. This was a perfect application of the Monte Carlo method because it
avoided both daunting analytical work and dangerous empirical work. During
this same era, Metropolis et al. (1953) showed that a new version of Monte
Carlo simulation based on Markov chains could model the movement of atomic
particles in a box when analytical calculations are impossible.
Most statistical computing tasks today are sufficiently routinized that many
scholars pay little attention to implementation details such as default settings,
methods of randomness, and alternative estimation techniques. The vast majority
of statistical software users blissfully point-and-click their way through machine
implementations of noncomplex procedures such as least squares regression,
cross-tabulation, and distributional summaries. However, an increasing number
of social scientists regularly use more complex and more demanding comput-
ing methods, such as Monte Carlo simulation, nonlinear estimation procedures,
queueing models, Bayesian stochastic simulation, and nonparametric estimation.
Accompanying these tools is a general concern about the possibility of knowingly
or unknowingly producing invalid results.
In a startling article, McCullough and Vinod (1999) find that econometric
software packages can still produce “horrendously inaccurate” results (p. 635)
Other documents randomly have
different content
of which would be the sky; the side walls folding screens, which may
be adjusted to any shape and size; the floor would be the grass
covered with rugs. On these rugs stand a few chairs, a couch, and a
small table. With such surroundings, altogether at home would Celia
seem, while she would say:
“Why, cousin; why Rosalind;—
Cupid have mercy!—Not a word?”
The many songs, especially “Under the Greenwood tree” and
“What shall He have that killed the Deer?” would prove very
appropriate in the forest of Arden environment, and the trees would
be quite in place for the love-verses of Orlando.
Or the guests might be served with a literary salad. Paste or draw
pictures on cards to illustrate the title of a book, and give one to
each person. Whoever shall make a correct guess without assistance
within fifteen minutes may be presented with a wreath of laurel.
This may be worn on the head or carried on the arm.
Sometimes a fête champêtre is given for sweet charity. It then
assumes a different phase, as booths, chalets, or tents are erected,
within which saleable articles are offered. An effective fête might be
given in athletic grounds, which should be noticeably gay with
streamers of bunting and little and big flags. At such a fête a large
orchestra should play the entire afternoon.
It would be very attractive if those in charge of the chalets would
represent milkmaids, as this allows picturesque apparel. The young
ladies might go bareheaded, or wear a gay handkerchief coquettishly
knotted under their braids or curls, or cover their heads altogether
by donning the new lawn sunbonnet, which is such a dainty feature
of this summer’s outing.
The chalets should be small lean-tos, their roofs tilting towards
the back and resting on four poles, one at each corner. These
chalets should be festively trimmed, and contain such products as
milk, cream, cheese, and eggs. As these are all necessities in
housekeeping, the financial result should be quite large.
Gowns and hats, flounces and ribbons, form a conspicuous part of
a fête champêtre. Sheer grenadines, nets, and gauzes, clouds of
Valenciennes lace, beflowered organdies, any of the effective
summer costumes, the more fetching the combination the more
satisfactory the attire. The color contrasts are allowed to a greater
extreme than for street apparel, and brilliant colors produce a smart
effect on the lawn; and yet the dainty white, yellow, pink, or blue
fabrics may be always afterwards worn to advantage, they are so
fresh and youthful.
The smart costume requires the broad-brimmed hat coquettishly
rolled, and massed with lilacs, morning-glories, sweet-pease, roses,
or carnations, and the often added long ribbon streamers. But the
flower toque, and the parasol of white mousseline de soie trimmed
with flowers and a flounce of lace, and the pretty or quaint fan, aid
the charming gown in producing an artistic effect.
The guests arrive in pony carriages, high carts, or victorias, and
the closed brougham, like an old friend, is always admissible. The
host and the hostess seem especially cordial, standing, as they do,
under the broad branches of a tall tree. Indeed, stern Madam
Propriety would deem such warmth of welcome scarcely permissible
under a lighted chandelier. But if, as it has been known to happen,
the day of the fête should also be the day of the worst storm of the
entire season, the guests are received, if possible, on the piazza, and
all aid in making merry and helping the hostess to such an extent
that people forget that a fête champêtre was ever considered, and
that it was not meant to be a house party from the beginning. Of
course no one should allude to the weather; that would be decidedly
out of form, and be very unkind to the hostess, who certainly cannot
stop the storm.
In such a shaping of events refreshments are served in-doors, if
possible using the same little tables intended for the lawn, the
cloths, which are edged about with ferns and field-flowers in variety,
added to the pretty china and cut glass used in serving the menu,
lend the charm of beauty.
The menu for such a function may be the same as that given at
an evening reception, or it may be the simple refreshment provided
for an afternoon tea, with an added salad or ice. But as an afternoon
spent in the open air gives good appetite, liberal refreshment will be
in order.
 THE KING’S CHILDREN.
“Pearly! Pearly!”
It was a woman’s shrill voice that fiercely shrieked the name out
into the morning air.
We were homeward bound from the Old Red Spring in Saratoga,
when we were arrested by her screams. The sun shone brightly, the
robins and other song birds were trilling out their sweetest melodies,
the air was heavily scented with white clover blossoms and
sweetbrier. It was a rarely beautiful July morning. All the world to us
was melody, save the jar made by this thin, haggard, unkempt
woman. In her effort to be heard she travelled along the road in the
direction she thought Pearly must have gone, crushing the daisies
and buttercups down before her.
Two Sabbaths before we had sat at the communion table, and
then felt a kinship to all, that our brothers and sisters were not only
those of our very own by ties of blood, but were close to us the
round world over. The Sabbath before, as the clergyman said, “freely
ye have received, freely give,” we thought more of the giving of
ourselves than of our money, more of letting others have a share of
the good gifts that had been our lot, joy, music, loving-kindness
generally, than of offering our filthy lucre. Indeed, it seemed a great
descent, for we had been taken up on the moment by our pastor’s
tender words, and now must remember Vanity Fair and the necessity
for money in this worldly world. And so thinking, this woman with
the wild, disagreeable voice, stopped us; and should we not do
something to help her, was the question put to ourselves.
She was one of the resident Saratogians. Cross, possibly, because
she was tired; haggard, because she had no time for rest. To her the
Spring waters were as a myth, and the dry, bracing air little
considered in her work-a-day existence. We, therefore, turning in the
direction in which this woman went, commenced our search for the
little girl, for such we decided she was, but all in vain. Whether
Pearly, familiar to the harsh voice and recognizing extra work or
disagreeable duty as a result of coming to the front, had hidden
behind the large clumps of elderberry bushes which grew thickly
around, or had run off to the woods for protection, we know not; we
only know that we had to leave the woman to conclude her search
alone. But the words, “Pearly! Pearly!” now and again caught our
ear, though indistinctly, as the distance widened between us, and
later we lost the sound altogether. Then it was that another Pearly
came into our thought.
She had been baptized Margaret; but the old-fashioned long
name had been shortened during her babyhood to the beautiful
name, “Pearl.” She has always been loving and lovable, and always
seemed consecrated, even from her cradle. Many of the wise people
have often gravely said of her, “That child can never live to grow up.
She is too good.” But she has lived to grow up, and, nothing
happening, in a year or two more she will be graduated from one of
our most respected women’s colleges. She, even as a little child,
never had to be punished. “Pearl, that is wrong; you should not act
or speak that way,” was the most serious chiding she ever needed to
receive; for when told she had done wrong, she would immediately
say, “I will try never to do that again.” And she invariably would keep
her promise.
As a schoolgirl she is a general favorite, being popular enough to
receive the unanimous vote for class president, for Pearl is a sunny,
bright, sympathetic girl. The truly good are always the truly happy.
Her religion is of the character to attract, not to repel. And possibly
there are nowhere to be found keener or more severe critics than
schoolgirls are of one another. The long-faced piety, as it is
sometimes called, would receive from them only ridicule and
contempt. The abandon of youth is not slow in exposing what they
consider trustless and wrong.
 But my story would be too long to tell many incidents in the life of
Pearl; to tell the many ways she has helped all with whom her short
life of eighteen years has brought her in contact; to tell of her
sympathetic words, helpful handclasp, feet swift to run on deeds of
kindness, voice raised in song, thus aiding others in the schoolroom,
the prayer-meeting or the home. Indeed, Pearl was constantly
forming new ties, thus binding the hearts of all who met her to
herself.
The incident of which I would particularly write is her work as a
King’s Daughter. She was one of the earliest to join this organization,
and the first band she formed was to pay for the education of a
young girl in the same school as herself. This young girl was the only
child of a rich father, but it was the old story—a dishonest partner
used the firm’s money for speculating purposes, and in an evil hour
all was gone; not only money, but reputation also, and Elsie, the
only child, must now leave school, it seemed, forever. Then it was
Pearl came to the rescue; and first binding her ten to secrecy,
because it would wound Elsie to ever know, it was arranged with the
President and officers of the school that this band should pay for
Elsie’s schooling; and she will graduate with Pearl, all unconscious of
the one to whom she is indebted.
Elsie’s father was notified by the school President that his
daughter was too much beloved not to have an opportunity to finish
her education. If he was ever able to refund the money, all right, if
not, it was still all right; and this is all Elsie or her father know.
Since then Pearl has started nine other bands, each doing noble
work for Christ and humanity. With only one of these is she herself
connected. It every year supports ten poor, aged women, who
otherwise would be obliged to go to the almshouse. By the help of
this King’s Daughters’ Band these women remain in their own little
homes, passing the hours as their desires dictate, and not feeling
the pain which Will Carleton so vividly describes in his poem, “Over
the Hills to the Poorhouse.” No wonder that these poor old women
frequently ask God’s blessing on these young girls, for they are so
comfortable and happy as they thus quietly wait for the summons to
the other home whose builder and maker is God. And no wonder
that Pearl wears a happy face, for the face indicates the heart
within. The good she has done, and may yet do, will never be known
here, nor is it necessary. Sufficient for Pearl will be the words which
we hope will also be ours some day, “Enter thou into the joy of thy
Lord.”
 FOR THE BOYS.
Why should not the boys be as busy and helpful as the girls?
Why should not the boys form their “Try Bands,” “Working Circles”
and “King’s Sons’ Societies?”
There is no reason. Will not, therefore, the willing, manly boys
who read this enlist their friends to help at least one of their heathen
brothers to a Christian education? It is work that will give abundant
reward.
American boys know how much care is taken for their education.
Not only are their teachers, but their mothers, fathers, brothers
sisters and other relatives, their constant instructors. The greatest
culture and opportunity surround them; valuable libraries are ever at
their disposal. There are numberless free schools, art rooms and
museums. Beside the private academies, institutes, and colleges,
there are Young Men’s Christian Association rooms, Christian
Endeavor Societies, churches and Sunday-schools, all open and
giving hearty welcome. Not so are the privileges of the boys in India,
China and Japan. For though the Bible and our missionaries have
done a great deal to help the heathen boy, his surroundings are dark
indeed, in contrast with those of children in Christian lands. Indeed,
it is rare that a heathen boy is not a castaway by his family when he
confesses Christ. Instead of relatives being a help to his life, they are
among his greatest sorrows.
Boys ought to be willing to give other boys a chance, especially
when they stand alone. If ten boys would form a band, they could
easily collect thirty dollars a year, and thirty dollars would pay all the
yearly expenses of a boy in a mission school or academy. This
academy prepares boys for the theological seminary, and the
seminary fits them for the ministry. Indeed, when in the academy,
boys often go out to talk and sing to those who do not know of
Christ. They feel sure that their heathen friends are missing so much
in not knowing Jesus, that they cannot wait until they have
completed their studies; but as soon as they know about Jesus
themselves they are impatient to tell others. They can talk to their
friends with greater effect than missionaries from this country,
because they understand their customs and ways. Besides, the
terrible heat in India does not affect them as it does people who go
from this country. Very often our missionaries and their families have
to return to America on account of their health.
Some of the boys in India are very bright. I will tell you of one
who is about fifteen years of age, and is a student in the Arcot
Academy, India. His name is Joseph, son of the catechist Israel; his
mother’s name is Rachel. You will notice they are all Bible names.
This family were once heathen, but now all know and love Christ,
and are happy in working for Him. I lately had the pleasure of
reading a letter written and composed by Joseph, without any aid
from his instructors. I wish it were possible for my boy readers to
see his penmanship; it seemed nearly as perfect as copperplate.
Each letter was very distinct and prettily shaded. Every word was
spelled correctly, and while his composition had not the exact style
we would use, it was very direct and intelligible. I doubt if many
American boys of Joseph’s age could do better with a French or
German letter, or in writing in any other language than their own.
Thinking you might be interested in hearing from Joseph, particularly
as he tells of what he does on the Sabbath, and of his school life, I
will quote directly from his letter:
“Madras Presidency, Ranipet, India.
“Most Respected and Kind Madam:
“My superiors, teachers and fellow-students are
doing well up to this time by the grace of our Almighty,
hoping the same for you....
 “I solicit you, dear madam, pray for me that I may
obey my superiors; I don’t like to have the name of
our Lord Jesus Christ in vain. But I want to publish His
name.
“Every Sunday we all go over the country and
preach about the Gospel. Many of the heathen become
Christians.
“There are eight bands in the school. When we are
going to preach, each band will take three or four
lyrics, some tracts, a cymbal, and a jalar or
tambourine. When we are singing many men and
women and children will come to hear us.
“After our preaching is over we take account of the
men, women and children who come to hear our
preaching. Most of them will ask questions, and we will
answer them. Many of them will abuse the name of
our Lord Jesus Christ. Though they abused Him, we
won’t leave these men, since they don’t understand
what Christ has done for the world. We have meeting
every Friday evening, and in that meeting we will give
our reports of the men who heard the Word of Christ.”
Referring to his studies, he writes:
“Now there are four classes, viz., matriculation
class, the fifth class, the lower secondary class, and
the lower fourth class. There are five teachers,
including our manager. Each class changes its lessons
after one hour. Our manager teaches general English
for the four classes, and also takes English history for
the fifth class, and science, physics and chemistry for
the sixth class. He shows many good examples and
gives us good games. He is very kind toward us. We
have many sorts of games. Tennis and football and
 cricket and gymnasium exercises. Our manager
teaches us cricket every evening.”
As this boy is writing to the one who supports him, he closes in
the following manner:
“I thank the Lord for having given me a supporter. I
render my warm and delightful obeisance to respected
and dear madam. I remain your most obedient
“Ponnor Isaac Joseph.”
After reading this letter, which is not a fancy sketch, picture in
your mind Joseph, his surroundings, a young Hindoo boy, whose
dark-skinned face glows enthusiastically with his love for Christ and
with his ardent desire to tell others of his love, writing in a strange
tongue to a lady whom he has never seen. He has her photograph,
and has received letters from her, but her voice and manner are only
conjectures in his mind. He is writing to this lady, who has been the
means of his salvation, of freeing him from his yoke borne by his
countrymen. Try to picture this, and then see if in your own heart
there is not a strong desire to free more than one boy in that dark
land. In freeing one, you free others: do not forget that.
 “I WISH I WERE A GENERAL.”
“If wishes were horses,
Beggars might ride.”
“Have you ever heard that, Jo?”
“Heard it, what kind of a bringing up has a fellow had, do you
think? You know well enough that ever since I was in
knickerbockers, that immortal rhyme has been drilled into me. I’m
sick and tired of sermonizing, and all I have to say is, if you don’t
wish for something grand, something beyond you, you never will
amount to anything.”
“That is true, Jo, but wishing without action will not accomplish
much. I’ve heard you make at least twenty wishes this morning.
One, ‘I wish I was rich!’ just as though that were anything new; all
boys wish that. Then you wished you were somebody great,
somebody famous, like Cæsar or the Czar of Russia, or the President
of the United States. Then you wished your father could only let you
have a college education so that you might be a lawyer. And then, to
go on to smaller matters, you wished it was Christmas, so that you
might have vacation. And lastly, you wished you were a fine bicycle
rider, so that you might win the prize in the coming race. I tell you,
old fellow, I long ago learned such a wholesome lesson on the
wishing point, that it made me over new, so to speak.”
“How so, John? now I am interested, for I thought you had been
perfect from your youth up.”
“Well, to begin with the beginning and make an out-and-out
confession, I’ll have to introduce you to my Uncle Charles. I wish you
knew General Journay; I know you would like him even if he is an
odd-looking man; he was once very handsome. He is too sensible to
think he is handsome now, though, for there is no denying that he’s
fat. He says it is constitutional, and maybe it is. I notice he is very
uncomfortable, short of breath, you know; gets a red face in
climbing up the stairs to the elevated road, and all that, but he’s jolly
and good, and says he wants me to be a manly man, and I am
going to try my best to please him. You know I am not as rich in
relations as you are, for my parents died when I was a baby, and I
never had either brothers or sisters; perhaps that’s one reason I
think so much of you, Jo. Well, to go on with my story, when I was
about twelve years old I went to visit for a week at my Uncle
Charles’ home. He was delighted to have me with him, and I never
tired of his companionship, or of looking at his soldier’s uniform, his
sword and his medals. One day I said to him, ‘Oh, Uncle, I wish I
were a General,’ and he replied, ‘There is no reason why you cannot
be one, my boy, if the right material is only in you.’”
“‘What do you mean by right material, Uncle?’ I inquired.
“‘Why, humility, obedience, courage, honesty, truthfulness.’
“‘I did not know that soldiers were ever humble.’
“‘You must be humble enough to enter the lowest ranks, obedient
enough to follow orders, courageous enough to face any emergency,
honest enough to submit to pain rather than to steal, and truthful
enough never to soil your lips or conscience with a lie.’
“Then my uncle told me of his own boyhood, of his poverty, his
hindrances, his temptations; and I saw that the rank of General did
not come by wishing, but by the greatest endurance, study, and
hard work. I tell you what, Jo, as I listened to his story I felt so
ashamed, and so small, I thought I would like to crawl away in a
hole, anywhere almost, if I could only hide, for you know my uncle is
such a noble, grand man. Then, too, my uncle told me of our great
inventors, officers, rulers, whom the world is delighted to honor, and
I saw that wishing had but little to do with their achievements and
successes. I saw I had to buckle on my own armor and go to work.
 “That night I could scarcely sleep; I kept thinking how
insignificant uncle must think me, for I knew I had often wished for
this, that and the other thing in his presence, and so when I did
sleep I dreamed that I was in the woods, and I thought that all the
bushes and trees were waving, and one big branch seemed like a
long, bare arm beckoning to me. I felt an awesome, queer, uncanny
feeling, and I was sure I was losing my way. I saw one and another
path, but which one to take I knew not, when suddenly I heard a
laugh; this frightened me so much that I jumped; then a voice said,
‘You little goosey-gander, what a brave soldier you would make, to
be sure, afraid of a little laugh;’ and then I heard ha! ha! ha! and
what seemed to me to be the most uproarious laughter, the shout of
a hundred fairies. Soon a tiny old woman approached me saying, ‘I
am a fairy queen. Ask for whatever you may wish while you are in
my domain?’
“At once I exclaimed: ‘I wish to be the oldest General living.’ And
there I was, a general in very truth, but so old I could scarcely see,
so deaf I could scarcely hear; and I was dressed in a costume similar
to my uncle’s. My hands were wrinkled, a long beard hung over my
breast, but it was as white as snow. My mouth felt so queer that I
lifted my hand to discover the reason, and alas! my teeth were all
gone. I tried to walk, but I was so stiff I could scarcely place one
foot before the other. ‘Oh, what a fool I have been,’ I thought. ‘If
only I were a boy again? Oh, Uncle Charles, Uncle Charles!’ I
screamed.
“‘Why, my boy, what is the matter, you were groaning and
moaning so in your sleep, I thought something must be wrong?’
were his words.
“Wasn’t I grateful, though, to find it was only a dream. It seemed
too good to be true, to learn that I was really a boy again, that life
was before, and not behind, me. I tell you, Jo, I could scarcely wait
for day to come, to get at positive work. And since that horrible
nightmare, which taught me the silliness of wishing, I have been a
changed boy, and I do not think I will ever fall into that purposeless
talk again. But you don’t like sermons, excuse me, Jo.”
“You are a good fellow, John; I should not be worthy of friendship
such as yours, if I did not benefit by what you have told me. I will
try to follow your example. What do you say to our both being
manly men?”
“Those words have the right ring.” And so saying the two friends
walked off arm in arm.
 A HEBREW CHRISTIAN.
Sydney Arnheim was a Jew. I say was, because he has thrown off
the yoke of the Jewish faith, and this little story will tell you how it
all came about.
Sydney is an only child; his parents are rich; his father, a famous
Wall Street broker, is a power among his kind; his mother also
belongs to a wealthy Hebrew family, and her refined taste and
education show clearly in her surroundings. Anything that appeals to
her exquisite judgment is purchased, so that Sydney’s home shows
everywhere the touch of elegance, as well as the fitness of
perfection. Sydney’s own room bears the print of her careful taste,
and yet prominent among the rich hangings and delicate furnishing
you see a Winchester rifle, a trout pole, also a buck’s antlers, a blue
crane, a kingfisher, and several other well-known birds, all so skilfully
prepared by the taxidermist that you could scarcely be blamed if you
thought them yet alive. Yes, Sydney is a regular boy, and loves to
keep trophies of his sport in sight, as well as his gun and trout pole.
He says at times, “It makes it seem as if I’m in the Adirondack
woods whenever I look at them, and simply lifting my rod recalls
some experiences with papa in the boat with our guide, or else
wading the streams with my tutor and drawing out big fish.” So
Sydney’s mother, knowing the comfort these recollections bring,
allows her son to place his sporting equipments just where and as
he pleases. Thus the room has an odd, menagerie style of
appearance. And thus the home of this Israelitish family everywhere
tells of comfort and luxury.
Sydney, from his babyhood, was carefully trained in the customs
and belief of his people. His mother, so frequently his companion,
constantly talked about the greatness of his lineage, and told him of
the patriarchs, legislators, warriors, singers and prophets which were
among the Hebrew people. She also told him of Abraham and
Moses, and of the covenant with Abraham at Jehovah-jireh, “In thy
seed shall all the nations of the earth be blessed,” and also what the
Lord said to Jacob at Bethel, “The land whereon thou liest, to thee
will I give it, and to thy seed.” And she daily urged her son to walk in
the footsteps and keep the covenants of his people, and always to
serve the Lord God of Israel. The rabbi also would frequently visit
their home, and Sydney was accustomed to talk to him, as well as to
receive the old man’s instruction and blessing. Sydney loved to hear
from him the stories of Moses, Aaron and Joshua; of the Tishbite
and his servant Elisha; of Solomon and the temple; of the son of
Jesse, David, the sweet singer, and of the promise of the Messiah to
come.
And so it was when Sydney had attained his seventeenth year
that when asked what he would most like for his birthday-gift he
replied, “The education that will fit me the most thoroughly to
preach, not alone to my people, but to win many erring ones,
believers in the ‘false Christ’ to turn to the true faith.” He was at this
time, therefore, and for seven anxious, wearisome months
afterwards, the Jewest of all Jews, a devoted follower of Moses, but
not of the Lamb.
About the time of the birthday above alluded to, there came for a
visit of three months a cousin of Sydney, a little girl about three
years his junior. She, young as she was, however, was a Christian,
and had the Easter previous publicly confessed Christ, and united
with the same church her parents had long attended.
The question might readily be asked, “Since she was Sydney’s
cousin, how was it that she was not also a believer in the Hebrew
faith?” The reason is this. Her mother was Sydney’s father’s sister,
and when very young, only seventeen years of age, had surprised
and grieved her family by stating she was about to marry a
gentleman who was not of her religion. Many words regarding the
matter were exchanged at the time, but they were useless in
preventing the marriage, and a year later she saddened her parents
yet more by renouncing her Hebrew faith, and connecting herself
with the church of her husband. He was a most exemplary
gentleman, however, and, notwithstanding his religion, his wife’s
people could not fail to have a most profound respect for him. So, in
time, the families visited back and forth, but the topic of religion was
never introduced. Sydney’s father would sometimes sigh wearily,
when talking of his sister, and say his hope was that she would yet
return to the belief of her forefathers, and that in time all must be
well.
So now the little Edith was to spend many weeks with her Jewish
relations, but she had been told to be careful about her attendance
at church, and ever watchful of her conduct, indeed to act in every
way as the child of the dear Christ whom she loved so well. Her
mother, however, before consigning her to her brother’s care, simply
said to her, “I will pray for you, dear, that your faith fail not,” and
“Have no fear, mamma,” was the sweet reply. “Jesus will be with me
in Uncle Nathan’s home, as well as in my own. Perhaps He will even
have work for me to do there. You know Dr. M—— last Sabbath
morning talked to us on the subject of missions, and said there were
many kinds, and while we should pray and work for the foreign and
domestic fields, we must also remember those of our own
household, indeed, all everywhere, who do not love the Saviour.” So
Edith’s mother had no fear for her child, and into the uncle’s home
there came a blessing, the measure of which cannot be counted by
any earthly rule or computation, for who can measure the joy of
even one soul turned to the Saviour?
Edith and Sydney were now great friends, for while they had
known each other always, the constant companionship led to the
warmest friendship, and they were therefore as good comrades as a
boy and girl cousin could well be. Neither religion nor any topic
bearing on it was ever discussed before Edith. She never attended
their church, nor they hers. When the Sabbath came she would
always be accompanied to the church door, and when the service
was out some member of the family would be found without waiting
to walk home with her, and during her entire visit neither by word or
action was she allowed to feel she stood apart from her Jewish
relations, and therefore the time passed all too rapidly onward. She
remembered her promise to her mother, and most earnestly she
prayed to God to direct and help her. She also prayed for her uncle
and aunt, imploring the Father in heaven to lead them into the light;
but particularly she prayed for her cousin Sydney. They were such
comrades, so nearly of an age, and yet she felt there was a great
gulf fixed, and therefore she constantly plead that he might learn of
the Christ, the Saviour.
One day, just after a most fervent prayer for Sydney, her cousin
approached her unobserved while she was reading from the New
Testament. He exclaimed, on seeing the title, “Why, Edith, I never
had a copy of that book in my hands. I should like to read yours
sometime, if you don’t mind.”
“Mind! why no, Sydney. Take it along with you now.”
And he did. When her cousin left, Edith prayed as she never had
before, beseeching the Father to let the scales drop from his eyes
and show unto him the Christ. And God did open the boy’s eyes. He
did not read through a glass darkly, but with clearest vision. The
brightest light fell on the divine Word, the light which later led to his
giving up his old Hebrew faith, and his acceptance of Jesus.
All did not come at once. At the first reading he was troubled,
anxious, but not satisfied. He had many old questions to settle; he
had much pride to put aside; he spent many hours, and at times
away into the night in prayer. But peace at last came, the peace
which he feels will endure until the day when he will see the King in
His beauty.
And now Sydney longs for the conversion of his parents, and of
all of his people. His wish is to preach Christ, and so do all in his
power to lead his brethren, the Jews, unto the everlasting joy of the
New Jerusalem.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com