Experience Seamless Full Ebook Downloads for Every Genre at textbookfull.

com

Art of Computer Programming The Combinatorial

Algorithms Volume 4B 1st Edition Donald Knuth

https://textbookfull.com/product/art-of-computer-
programming-the-combinatorial-algorithms-volume-4b-1st-
edition-donald-knuth/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Extensions of Dynamic Programming for Combinatorial

Optimization and Data Mining Hassan Aboueisha

https://textbookfull.com/product/extensions-of-dynamic-programming-
for-combinatorial-optimization-and-data-mining-hassan-aboueisha/

textboxfull.com

Intelligent Algorithms in Software Engineering:

Proceedings of the 9th Computer Science On-line Conference
2020, Volume 1 Radek Silhavy
https://textbookfull.com/product/intelligent-algorithms-in-software-
engineering-proceedings-of-the-9th-computer-science-on-line-
conference-2020-volume-1-radek-silhavy/
textboxfull.com

Algorithms Illuminated Part 3 Greedy Algorithms and

Dynamic Programming 1st Edition Tim Roughgarden

https://textbookfull.com/product/algorithms-illuminated-part-3-greedy-
algorithms-and-dynamic-programming-1st-edition-tim-roughgarden/

textboxfull.com

The Art and Science of NFC Programming 1st Edition Anne-

Marie Lesas

https://textbookfull.com/product/the-art-and-science-of-nfc-
programming-1st-edition-anne-marie-lesas/

textboxfull.com
Batchography The Art of Batch Files Programming Elias
Bachaalany

https://textbookfull.com/product/batchography-the-art-of-batch-files-
programming-elias-bachaalany/

textboxfull.com

Guide to Competitive Programming: Learning and Improving

Algorithms Through Contests (Undergraduate Topics in
Computer Science) Antti Laaksonen
https://textbookfull.com/product/guide-to-competitive-programming-
learning-and-improving-algorithms-through-contests-undergraduate-
topics-in-computer-science-antti-laaksonen/
textboxfull.com

Algorithms For Competitive Programming 1st Edition David

Esparza Alba

https://textbookfull.com/product/algorithms-for-competitive-
programming-1st-edition-david-esparza-alba/

textboxfull.com

Algorithms Illuminated Part 3 Greedy Algorithms and

Dynamic Programming Tim Roughgarden

https://textbookfull.com/product/algorithms-illuminated-part-3-greedy-
algorithms-and-dynamic-programming-tim-roughgarden/

textboxfull.com

Combinatorial Algorithms 27th International Workshop IWOCA

2016 Helsinki Finland August 17 19 2016 Proceedings 1st
Edition Veli Mäkinen
https://textbookfull.com/product/combinatorial-algorithms-27th-
international-workshop-iwoca-2016-helsinki-finland-
august-17-19-2016-proceedings-1st-edition-veli-makinen/
textboxfull.com
This page intentionally left blank
THE ART OF
COMPUTER PROGRAMMING

-1
DONALD E. KNUTH Stanford University

 ADDISON–WESLEY

-2
Volume 4B / Combinatorial Algorithms, Part 2

THE ART OF
COMPUTER PROGRAMMING

Boston · Columbus · New York · San Francisco · Amsterdam · Cape Town

Dubai · London · Madrid · Milan · Munich · Paris · Montréal · Toronto · Delhi · Mexico City
São Paulo · Sydney · Hong Kong · Seoul · Singapore · Taipei · Tokyo

-3
Lyrics have been quoted on page 65 from the songs “Mississippi Mud,” written by
Harry Barris and James Cavanaugh, and “Pick Yourself Up,” written by Dorothy
Fields and Jerome Kern. Used by permission of Shapiro, Bernstein & Co, Inc.
The author and publisher have taken care in the preparation of this book, but make no
expressed or implied warranty of any kind and assume no responsibility for errors or
omissions. No liability is assumed for incidental or consequential damages in connection
with or arising out of the use of the information or programs contained herein.
For government sales inquiries, please contact governmentsales@pearsoned.com
For questions about sales outside the U.S., please contact intlcs@pearson.com
Visit us on the Web: informit.com/aw
Library of Congress Cataloging-in-Publication Data
Knuth, Donald Ervin, 1938-
The art of computer programming / Donald Ervin Knuth.
xviii,714 p. 24 cm.
Includes bibliographical references and index.
Contents: v. 1. Fundamental algorithms. -- v. 2. Seminumerical
algorithms. -- v. 3. Sorting and searching. -- v. 4a. Combinatorial
algorithms, part 1. -- v. 4b. Combinatorial algorithms, part 2.
Contents: v. 4b. Combinatorial algorithms, part 2.
ISBN 978-0-201-89683-1 (v. 1, 3rd ed.)
ISBN 978-0-201-89684-8 (v. 2, 3rd ed.)
ISBN 978-0-201-89685-5 (v. 3, 2nd ed.)
ISBN 978-0-201-03804-0 (v. 4a)
ISBN 978-0-201-03806-4 (v. 4b)
1. Electronic digital computers--Programming. 2. Computer
algorithms. I. Title.
QA76.6.K64 1997
005.1--DC21 97-2147

Internet page http://www-cs-faculty.stanford.edu/~knuth/taocp.html contains

current information about this book and related books.
See also http://www-cs-faculty.stanford.edu/~knuth/sgb.html for information
about The Stanford GraphBase, including downloadable software for dealing with
the graphs used in many of the examples.
And see http://www-cs-faculty.stanford.edu/~knuth/mmix.html for basic infor-
mation about the MMIX computer.
Copyright c 2023 by Pearson Education, Inc.
All rights reserved. This publication is protected by copyright, and permission must be
obtained from the publisher prior to any prohibited reproduction, storage in a retrieval
system, or transmission in any form or by any means, electronic, mechanical, photo-
copying, recording, or likewise. For information regarding permissions, request forms,
and the appropriate contacts with the Pearson Education Global Rights & Permissions
Department, please visit www.pearson.com/permissions/.
ISBN-13 978-0-201-03806-4
ISBN-10 0-201-03806-4
Second printing, November 2022
ScoutAutomatedPrintCode
 PREFACE
Begin at the beginning, and do not allow yourself to gratify
a mere idle curiosity by dipping into the book, here and there.
This would very likely lead to your throwing it aside,
with the remark “This is much too hard for me!,”
and thus losing the chance of adding a very large item
to your stock of mental delights.
— LEWIS CARROLL, in Symbolic Logic (1896)
Combinatorial algorithms are the methods that allow us to cope with
problems that involve zillions of cases. The explosive growth in the knowledge
of such techniques has meant that several volumes are needed to describe them.
Thus my original plan to devote Volume 4 of The Art of Computer Programming
to combinatorial algorithms has morphed into a plan to prepare Volumes 4A,
4B, and so on. This book is the second of that series, a sequel to Volume 4A.
In the preface to Volume 4A I explained why I was captivated by combina-
torial algorithms soon after I fell in love with computers. “The art of writing
such programs is especially important and appealing because a single good idea
can save years or even centuries of computer time.”
Chapter 7 began in Volume 4A with a short review of graph theory and
a longer discussion of “Zeros and Ones” (Section 7.1). That volume concluded
with Section 7.2.1, “Generating Basic Combinatorial Patterns,” which was the
first part of Section 7.2, “Generating All Possibilities.” Now the story continues,
with the opening parts of Section 7.2.2, “Backtrack Programming.”
Backtracking is the name for an important body of techniques that have been
a mainstay of combinatorial algorithms since the beginning. More than a third
of this book is devoted to Section 7.2.2.1, which explores data structures whose
links perform delightful dances. Such structures are ideally suited to backtrack
programming in general, and to the “exact cover problem” (XC) in particular.
The XC problem, also known as “set partitioning,” essentially asks for all ways
to cover a set of items, by choosing appropriate subsets of items called options.
Dozens of important applications turn out to be special cases of XC, and the
method of choice for such problems is often to use dancing links.
While writing this material I learned to my surprise that an apparently
innocuous extension of the classical XC problem leads to an enormous increase
in the number of significant special cases. This extended problem, called XCC
(for “exact covering with colors”), allows some of the items to receive various
colors. Colored items are allowed to be covered by many different options, as
long as the colors are compatible.
v

-5
vi PREFACE
Spoiler alert: With dancing links, we can solve XCC problems almost as
easily as XC problems! Therefore I believe that the study of XCC solvers, now
in its infancy, is destined to become quite important, and I’ve done my best to
introduce the subject here. There also are related methods for an even more
general class of problems called MCC (“multiple covering with colors”), and for
finding XCC solutions of minimum cost.
If you turn to a random page of Section 7.2.2.1, chances are good that you’ll
find some sort of puzzle being discussed. The reason is that puzzles are by far
the best means I know to illustrate the algorithms and techniques that are being
introduced here. The point of a puzzle is easily grasped; and the fact that an
extraordinary number of quite different puzzles all turn out to be special cases of
XCC and MCC is significant in itself. Indeed, it becomes clear that the same ideas
will solve many complex and harder-to-explain problems of the “real world.”
The new tools provided by dancing links allow me to emphasize the process
of creating new puzzles, rather than simply to explain how to resolve puzzles
that have already been posed. I’ve also tried my best to discuss the history
of each puzzle type, and to give credit to the brilliant innovators who created
them. As a result, I’m pleased that this book now contains, as a side-product of
my attempts to teach computer methods, a treasure trove of information about
recreational mathematics — from popular classics like edge-matching puzzles, or
queen placement, or polyominoes, or the Soma cube, or rectangle dissections,
or intriguing patterns of interlocking words, to more recent crazes like sudoku,
slitherlink, masyu, and hitori.
I’ve had loads of fun writing other parts of these volumes, but without
doubt Section 7.2.2.1 has been the funnest. And I know that my delight in good
puzzles is shared by a significant number of leading computer scientists and
mathematicians, who have told me that they chose their careers after having
been inspired by such intellectual challenges.
Knuth likes to include in those books [The Art of Computer Programming]
as much recreational material as he can cram in.
— MARTIN GARDNER, Undiluted Hocus-Pocus (2013)
The second half of this book is devoted to Section 7.2.2.2, “Satisfiability,”
which addresses one of the most fundamental problems in all of computer science:
Given a Boolean function, can its variables be set to at least one pattern of 0s
and 1s that will make the function true? This problem arises so often, people
have given it a nickname, ‘SAT’.
Satisfiability might seem like an abstract exercise in understanding formal
systems, but the truth is far different: Revolutionary methods for solving SAT
problems emerged at the beginning of the twenty-first century, and they’ve led to
game-changing applications in industry. These so-called “SAT solvers” can now
routinely find solutions to practical problems that involve millions of variables
and were thought until very recently to be hopelessly difficult.
Satisfiability is important chiefly because Boolean algebra is so versatile.
Almost any problem can be formulated in terms of basic logical operations, and

-6
 PREFACE vii
the formulation is particularly simple in a great many cases. Section 7.2.2.2
therefore begins with ten typical examples of widely different applications, and
closes with detailed empirical results for a hundred different benchmarks. The
great variety of these problems — all of which are special cases of SAT — is
illustrated on pages 300 and 301 (which are my favorite pages in this book).
The story of satisfiability is the tale of a triumph of software engineering,
blended with rich doses of beautiful mathematics. Section 7.2.2.2 explains how
such a miracle occurred, by presenting complete details of seven SAT solvers,
ranging from the small-footprint methods of Algorithms A and B to the indus-
trial strength, state-of-the-art methods of Algorithms W, L, and C. (Well I
have to hedge a little: New techniques are continually being discovered; hence
SAT technology is ever-growing and the story is ongoing. But I do think that
Algorithms W, L, and C compare reasonably well with the best algorithms of
their class that were known in 2010. They’re no longer at the cutting edge, but
they still are amazingly good.)
Wow — Sections 7.2.2.1 and 7.2.2.2 have turned out to be the longest sec-
tions, by far, in The Art of Computer Programming — especially Section 7.2.2.2.
The SAT problem is evidently a killer app, because it is key to the solution of so
many other problems. Consequently I can only hope that my lengthy treatment
does not also kill off my faithful readers! As I wrote this material, one topic
always seemed to flow naturally into another, so there was no neat way to break
either section up into separate subsections. (And anyway the format of TAOCP
doesn’t allow for a Section 7.2.2.1.3 or a Section 7.2.2.2.6.)
I’ve tried to ameliorate the reader’s navigation problem by adding sub-
headings at the top of each right-hand page. Furthermore, as always, the exer-
cises appear in an order that roughly parallels the order in which corresponding
topics are taken up in the text. Numerous cross-references are provided between
text, exercises, and illustrations, so that you have a fairly good chance of keeping
in sync. I’ve also tried to make the index as comprehensive as possible.
Look, for example, at a “random” page — say page 264, which is part of the
subsection about Monte Carlo algorithms. On that page you’ll see that exercises
302, 303, 299, and 306 are mentioned. So you can guess that the main exercises
about Monte Carlo algorithms are numbered in the early 300s. (Indeed, exer-
cise 306 deals with the important special case of “Las Vegas algorithms”; and the
next exercises explore a fascinating concept called “reluctant doubling.”) This
entire book is full of surprises and tie-ins to other aspects of computer science.
As in previous volumes, sections and subsections of the text are occasion-
ally preceded by an asterisk (∗), meaning that the topics discussed there are
“advanced” and skippable on a first reading.
You might think that a 700-page book has probably been padded with
peripheral material. But I constantly had to “cut, cut, cut” while writing it,
because a great deal more is known! I found that new and potentially interesting-
yet-unexplored topics kept popping up, more than enough to fill a lifetime; yet
I knew that I must move on. So I hope that I’ve selected for treatment here a
significant fraction of the concepts that will be the most important as time passes.

-7
viii PREFACE
Every week I’ve been coming across fascinating new things
that simply cry out to be part of The Art.
— DONALD E. KNUTH (2008)
Most of this book is self-contained, although there are frequent tie-ins with
the topics discussed in previous volumes. Low-level details of machine language
programming have already been covered extensively; so the algorithms in the
present book are usually specified only at an abstract level, independent of
any machine. However, some aspects of combinatorial programming are heavily
dependent on low-level details that didn’t arise before; in such cases, all examples
in this book are based on the MMIX computer, which supersedes the MIX machine
that was defined in early editions of Volume 1. Details about MMIX appear in a
paperback supplement to that volume called The Art of Computer Programming,
Volume 1, Fascicle 1, containing Sections 1.3.1´, 1.3.2´, etc.; they’re also available
on the Internet, together with downloadable assemblers and simulators.
Another downloadable resource, a collection of programs and data called The
Stanford GraphBase, is cited extensively in the examples of this book. Readers
are encouraged to play with it, in order to learn about combinatorial algorithms
in what I think will be the most efficient and most enjoyable way.
I wrote nearly a thousand computer programs while preparing this material,
because I find that I don’t understand things unless I try to program them. Most
of those programs were quite short, of course; but several of them are rather
substantial, and possibly of interest to others. Therefore I’ve made a selection
available by listing some of them on the following webpage:
http://www-cs-faculty.stanford.edu/~knuth/programs.html
In particular you can download the programs DLX1, DLX2, DLX3, DLX5, DLX6,
and DLX-PRE, which are the experimental versions of Algorithms X, C, M, C$,
Z, and P, respectively, that were my constant companions while writing Section
7.2.2.1. Similarly, SAT0, SAT0W, SAT8, SAT9, SAT10, SAT11, SAT11K, SAT13
are the equivalents of Algorithms A, B, W, S, D, L, L , C, respectively, in Section
7.2.2.2. Such programs will be useful for solving many of the exercises, if you
don’t have access to other XCC solvers or SAT solvers. You can also download
SATexamples.tgz from that page; it’s a collection of programs that generate data
for all 100 of the benchmark examples discussed in the text, and many more.
Several exercises involve the lists of English words that I’ve used in preparing
examples. You’ll need the data from
http://www-cs-faculty.stanford.edu/~knuth/wordlists.tgz
if you have the courage to work the exercises that use such lists.
Special Note: During the years that I’ve been preparing Volume 4, I’ve
often run across basic techniques of probability theory that I would have put
into Section 1.2 of Volume 1 if I’d been clairvoyant enough to anticipate them in
the 1960s. Finally I realized that I ought to collect most of them together in one
place, because the story of those developments is too interesting to be broken up
into little pieces scattered here and there.

-8
 PREFACE ix
Therefore this book begins with a special tutorial and review of probability
theory, in an unnumbered section entitled “Mathematical Preliminaries Redux.”
References to its equations and exercises use the abbreviation ‘MPR’. (Think of
the word “improvement.”)
Incidentally, just after the special MPR section, Section 7.2.2 begins inten-
tionally on a left-hand page; and its illustrations are numbered beginning with
Fig. 68. The reason is that Section 7.2.1 ended in Volume 4A on a right-hand
page, and its final illustration was Fig. 67. My editor has decided to treat Chap-
ter 7 as a single unit, even though it is being split into several physical volumes.
Special thanks are due to Nikolai Beluhov, Armin Biere, Niklas Eén, Mar-
ijn Heule, Holger Hoos, Wei-Hwa Huang, Svante Janson, Ernst Schulte-Geers,
George Sicherman, Filip Stappers, and Udo Wermuth, for their detailed com-
ments on my early attempts at exposition, as well as to dozens and dozens of
other correspondents who have contributed crucial corrections. My editor at
Addison–Wesley, Mark Taub, has expertly shepherded this series of books into
the 21st century; and Julie Nahil, as senior content producer, has meticulously
ensured that the highest publication standards have continued to be maintained.
Thanks also to Tomas Rokicki for keeping my Dell workstation in shipshape
order, as well as to Stanford’s InfoLab for providing extra computer power when
that machine had reached its limits.
I happily offer a “finder’s fee” of $2.56 for each error in this book when it is
first reported to me, whether that error be typographical, technical, or historical.
The same reward holds for items that I forgot to put in the index. And valuable
suggestions for improvements to the text are worth 32/c each. (Furthermore, if
you find a better solution to an exercise, I’ll actually do my best to give you
immortal glory, by publishing your name in subsequent printings:−)
Happy reading!
Stanford, California D. E. K.
June 2022

A note on references. Several oft-cited journals and conference proceedings

have special code names, which appear in the Index and Glossary at the close
of this book. But the various kinds of IEEE Transactions are cited by including
a letter code for the type of transactions, in boldface preceding the volume
number. For example, ‘IEEE Trans. C-35’ means the IEEE Transactions on
Computers, volume 35. The IEEE no longer uses these convenient letter codes,
but the codes aren’t too hard to decipher: ‘EC’ once stood for “Electronic
Computers,” ‘IT’ for “Information Theory,” ‘PAMI’ for “Pattern Analysis and
Machine Intelligence,” and ‘SE’ for “Software Engineering,” etc.; ‘CAD’ meant
“Computer-Aided Design of Integrated Circuits and Systems.”
A cross-reference such as ‘exercise 7.10–00’ points to a future exercise in
Section 7.10 whose number is not yet known.

-9
x PREFACE
A note on notations. Simple and intuitive conventions for the algebraic rep-
resentation of mathematical concepts have always been a boon to progress, espe-
cially when most of the world’s researchers share a common symbolic language.
The current state of affairs in combinatorial mathematics is unfortunately a bit
of a mess in this regard, because the same symbols are occasionally used with
completely different meanings by different groups of people; some specialists who
work in comparatively narrow subfields have unintentionally spawned conflicting
symbolisms. Computer science — which interacts with large swaths of math-
ematics — needs to steer clear of this danger by adopting internally consistent
notations whenever possible. Therefore I’ve often had to choose among a number
of competing schemes, knowing that it will be impossible to please everyone.
I have tried my best to come up with notations that I believe will be best for the
future, often after many years of experimentation and discussion with colleagues,
often flip-flopping between alternatives until finding something that works well.
Usually it has been possible to find convenient conventions that other people
have not already coopted in contradictory ways.
Appendix B is a comprehensive index to all of the principal notations that
are used in the present book, inevitably including several that are not (yet?)
standard. If you run across a formula that looks weird and/or incomprehensible,
chances are fairly good that Appendix B will direct you to a page where my
intentions are clarified. But I might as well list here a few instances that you
might wish to watch for when you read this book for the first time:
• Hexadecimal constants are preceded by a number sign or hash mark. For
example, # 123 means (123)16 .
• The “monus” operation x − . y, sometimes called dot-minus or saturating
subtraction, yields max(0, x − y).
• The median of three numbers {x, y, z } is denoted by xyz .
• The “two dots” notations (x . . y ), (x . . y ], [x . . y ), and [x . . y ] are used to
denote intervals.
• A set such as {x}, which consists of a single element, is often denoted simply
by x in contexts such as X ∪ x or X \ x.
• If n is a nonnegative integer, the number of 1-bits in n’s binary representation
is νn. Furthermore, if n > 0, the leftmost and rightmost 1-bits of n are
respectively 2λn and 2 ρn . For example, ν 10 = 2, λ 10 = 3, ρ 10 = 1.
• The Cartesian product of graphs G and H is denoted by G H. For example,
Cm Cn denotes an m × n torus, because Cn denotes a cycle of n vertices.

-10
 NOTES ON THE EXERCISES

The exercises in this set of books have been designed for self-study as well
as for classroom study. It is difficult, if not impossible, for anyone to learn a
subject purely by reading about it, without applying the information to specific
problems and thereby being encouraged to think about what has been read.
Furthermore, we all learn best the things that we have discovered for ourselves.
Therefore the exercises form a major part of this work; a definite attempt has
been made to keep them as informative as possible and to select problems that
are enjoyable as well as instructive.
In many books, easy exercises are found mixed randomly among extremely
difficult ones. A motley mixture is, however, often unfortunate because readers
like to know in advance how long a problem ought to take — otherwise they
may just skip over all the problems. A classic example of such a situation is
the book Dynamic Programming by Richard Bellman; this is an important,
pioneering work in which a group of problems is collected together at the end
of some chapters under the heading “Exercises and Research Problems,” with
extremely trivial questions appearing in the midst of deep, unsolved problems.
It is rumored that someone once asked Dr. Bellman how to tell the exercises
apart from the research problems, and he replied, “If you can solve it, it is an
exercise; otherwise it’s a research problem.”
Good arguments can be made for including both research problems and
very easy exercises in a book of this kind; therefore, to save the reader from
the possible dilemma of determining which are which, rating numbers have been
provided to indicate the level of difficulty. These numbers have the following
general significance:
Rating Interpretation
00 An extremely easy exercise that can be answered immediately if the
material of the text has been understood; such an exercise can almost
always be worked “in your head,” unless you’re multitasking.
10 A simple problem that makes you think over the material just read, but
is by no means difficult. You should be able to do this in one minute at
most; pencil and paper may be useful in obtaining the solution.
20 An average problem that tests basic understanding of the text mate-
rial, but you may need about fifteen or twenty minutes to answer it
completely. Maybe even twenty-five.
xi

-11
xii NOTES ON THE EXERCISES
30 A problem of moderate difficulty and/or complexity; this one may
involve more than two hours’ work to solve satisfactorily, or even more
if the TV is on.
40 Quite a difficult or lengthy problem that would be suitable for a term
project in classroom situations. A student should be able to solve the
problem in a reasonable amount of time, but the solution is not trivial.
50 A research problem that has not yet been solved satisfactorily, as far
as the author knew at the time of writing, although many people have
tried. If you have found an answer to such a problem, you ought to
write it up for publication; furthermore, the author of this book would
appreciate hearing about the solution as soon as possible (provided that
it is correct).
By interpolation in this “logarithmic” scale, the significance of other rating
numbers becomes clear. For example, a rating of 17 would indicate an exercise
that is a bit simpler than average. Problems with a rating of 50 that are subse-
quently solved by some reader may appear with a 40 rating in later editions of
the book, and in the errata posted on the Internet (see page iv).
The remainder of the rating number divided by 5 indicates the amount of
detailed work required. Thus, an exercise rated 24 may take longer to solve
than an exercise that is rated 25, but the latter will require more creativity. All
exercises with ratings of 46 or more are open problems for future research, rated
according to the number of different attacks that they’ve resisted so far.
The author has tried earnestly to assign accurate rating numbers, but it is
difficult for the person who makes up a problem to know just how formidable it
will be for someone else to find a solution; and everyone has more aptitude for
certain types of problems than for others. It is hoped that the rating numbers
represent a good guess at the level of difficulty, but they should be taken as
general guidelines, not as absolute indicators.
This book has been written for readers with varying degrees of mathematical
training and sophistication; as a result, some of the exercises are intended only for
the use of more mathematically inclined readers. The rating is preceded by an M
if the exercise involves mathematical concepts or motivation to a greater extent
than necessary for someone who is primarily interested only in programming
the algorithms themselves. An exercise is marked with the letters “HM ” if its
solution necessarily involves a knowledge of calculus or other higher mathematics
not developed in this book. An “HM ” designation does not necessarily imply
difficulty.
Some exercises are preceded by an arrowhead, “ ”; this designates prob-
lems that are especially instructive and especially recommended. Of course, no
reader/student is expected to work all of the exercises, so those that seem to
be the most valuable have been singled out. (This distinction is not meant to
detract from the other exercises!) Each reader should at least make an attempt
to solve all of the problems whose rating is 10 or less; and the arrows may help
to indicate which of the problems with a higher rating should be given priority.

-12
 NOTES ON THE EXERCISES xiii
Several sections have more than 100 exercises. How can you find your way
among so many? In general the sequence of exercises tends to follow the sequence
of ideas in the main text. Adjacent exercises build on each other, as in the
pioneering problem books of Pólya and Szegő. The final exercises of a section
often involve the section as a whole, or introduce supplementary topics.
Solutions to most of the exercises appear in the answer section. Please use
them wisely; do not turn to the answer until you have made a genuine effort to
solve the problem by yourself, or unless you absolutely do not have time to work
this particular problem. After getting your own solution or giving the problem a
decent try, you may find the answer instructive and helpful. The solution given
will often be quite short, and it will sketch the details under the assumption
that you have earnestly tried to solve it by your own means first. Sometimes the
solution gives less information than was asked; often it gives more. It is quite
possible that you may have a better answer than the one published here, or you
may have found an error in the published solution; in such a case, the author
will be pleased to know the details. Later printings of this book will give the
improved solutions together with the solver’s name where appropriate.
When working an exercise you may generally use the answers to previous
exercises, unless specifically forbidden from doing so. The rating numbers have
been assigned with this in mind; thus it is possible for exercise n + 1 to have a
lower rating than exercise n, even though it includes the result of exercise n as
a special case.

Summary of codes: 00 Immediate

10 Simple (one minute)
20 Medium (quarter hour)
 Recommended 30 Moderately hard
M Mathematically oriented 40 Term project
HM Requiring “higher math” 50 Research problem

EXERCISES
 1. [00 ] What does the rating “M15 ” mean?
2. [10 ] Of what value can the exercises in a textbook be to the reader?
3. [HM45 ] Prove that every simply connected, closed 3-dimensional manifold is topo-
logically equivalent to a 3-dimensional sphere.

The men that stood for office, noted for acknowledged worth,
And for manly deeds of honour, and for honourable birth;
Train’d in exercise and art, in sacred dances and in song,
All are ousted and supplanted by a base ignoble throng.
— ARISTOPHANES, The Frogs (405 B.C.)

-13
xiv NOTES ON THE EXERCISES

Here mine aduice, shall be to those Artificers that will profite in this,
or any of my bookes nowe published, or that hereafter shall be,
firste confusely to reade them thorow; then with more iudgement,
and at the thirde readinge wittely to practise. So fewe thinges shall be vnknowen.
— LEONARDE DIGGES, A Boke named Tectonicon (1556)
Now I saw, tho’ too late, the Folly of
beginning a Work before we count the Cost,
and before we judge rightly of our own Strength to go through with it.
— DANIEL DEFOE, Robinson Crusoe (1719)

-14
 CONTENTS

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Notes on the Exercises . . . . . . . . . . . . . . . . . . . . . . . xi
Mathematical Preliminaries Redux . . . . . . . . . . . . . . . . . . 1
Inequalities . . . . . . . . . . . . . . . . . . . . . . 3
Martingales . . . . . . . . . . . . . . . . . . . . . . 6
Tail inequalities from martingales . . . . . . . . . . . . . 8
Applications . . . . . . . . . . . . . . . . . . . . . . 9
Statements that are almost sure, or even quite sure . . . . . 11
Exercises . . . . . . . . . . . . . . . . . . . . . . . 12
Chapter 7 — Combinatorial Searching . . . . . . . . . . . . . . . . 0
7.2. Generating All Possibilities . . . . . . . . . . . . . . . . . . . . 0
7.2.1. Generating Basic Combinatorial Patterns . . . . . . . . . . . 0
7.2.2. Backtrack Programming . . . . . . . . . . . . . . . . . . . 30
Data structures . . . . . . . . . . . . . . . . . . . . . 32
Walker’s method . . . . . . . . . . . . . . . . . . . . 33
Permutations and Langford pairs . . . . . . . . . . . . . 34
Word rectangles . . . . . . . . . . . . . . . . . . . . 36
Commafree codes . . . . . . . . . . . . . . . . . . . . 37
Dynamic ordering of choices . . . . . . . . . . . . . . . 38
Sequential allocation redux . . . . . . . . . . . . . . . . 39
Lists for the commafree problem . . . . . . . . . . . . . 41
A general mechanism for doing and undoing . . . . . . . . 43
Backtracking through commafree codes . . . . . . . . . . 44
Running time estimates . . . . . . . . . . . . . . . . . 46
*Estimating the number of solutions . . . . . . . . . . . . 49
Factoring the problem . . . . . . . . . . . . . . . . . . 52
Historical notes . . . . . . . . . . . . . . . . . . . . . 53
Exercises . . . . . . . . . . . . . . . . . . . . . . . 55
7.2.2.1. Dancing links . . . . . . . . . . . . . . . . . . . 65
Exact cover problems . . . . . . . . . . . . . . . . . . 66
Secondary items . . . . . . . . . . . . . . . . . . . . 70
Progress reports . . . . . . . . . . . . . . . . . . . . 73
Sudoku . . . . . . . . . . . . . . . . . . . . . . . . 74
Polyominoes . . . . . . . . . . . . . . . . . . . . . . 79
Polycubes . . . . . . . . . . . . . . . . . . . . . . . 82
Factoring an exact cover problem . . . . . . . . . . . . . 83
xv

-15
xvi CONTENTS

Color-controlled covering . . . . . . . . . . . . . . . . . 87
Introducing multiplicity . . . . . . . . . . . . . . . . . 92
*A new dance step . . . . . . . . . . . . . . . . . . . . 95
*Analysis of Algorithm X . . . . . . . . . . . . . . . . . 98
*Analysis of matching problems . . . . . . . . . . . . . . 102
*Maintaining a decent focus . . . . . . . . . . . . . . . . 104
Exploiting local equivalence . . . . . . . . . . . . . . . 106
*Preprocessing the options . . . . . . . . . . . . . . . . 108
Minimum-cost solutions . . . . . . . . . . . . . . . . . 111
*Implementing the min-cost cutoffs . . . . . . . . . . . . . 116
*Dancing with ZDDs . . . . . . . . . . . . . . . . . . . 119
Summary . . . . . . . . . . . . . . . . . . . . . . . 122
Historical notes . . . . . . . . . . . . . . . . . . . . . 123
Exercises — First set . . . . . . . . . . . . . . . . . . . 124
Exercises — Second set . . . . . . . . . . . . . . . . . . 156
Exercises — Third set . . . . . . . . . . . . . . . . . . 174
7.2.2.2. Satisfiability . . . . . . . . . . . . . . . . . . . . 185
Example applications . . . . . . . . . . . . . . . . . . 188
Backtracking algorithms . . . . . . . . . . . . . . . . . 211
Random clauses . . . . . . . . . . . . . . . . . . . . 231
Resolution of clauses . . . . . . . . . . . . . . . . . . 238
Clause-learning algorithms . . . . . . . . . . . . . . . . 244
Monte Carlo algorithms . . . . . . . . . . . . . . . . . 261
The Local Lemma . . . . . . . . . . . . . . . . . . . 265
*Message-passing algorithms . . . . . . . . . . . . . . . 274
*Preprocessing of clauses . . . . . . . . . . . . . . . . . 279
Encoding constraints into clauses . . . . . . . . . . . . . 281
Unit propagation and forcing . . . . . . . . . . . . . . . 287
Symmetry breaking . . . . . . . . . . . . . . . . . . . 289
Satisfiability-preserving maps . . . . . . . . . . . . . . . 291
One hundred test cases . . . . . . . . . . . . . . . . . 297
Tuning the parameters . . . . . . . . . . . . . . . . . . 308
Exploiting parallelism . . . . . . . . . . . . . . . . . . 312
History . . . . . . . . . . . . . . . . . . . . . . . . 313
Exercises . . . . . . . . . . . . . . . . . . . . . . . 317

-16
 CONTENTS xvii

Answers to Exercises . . . . . . . . . . . . . . . . . . . . . . . . 370

Appendix A — Tables of Numerical Quantities . . . . . . . . . . . . 656
1. Fundamental Constants (decimal) . . . . . . . . . . . . . . 656
2. Fundamental Constants (hexadecimal) . . . . . . . . . . . . 657
3. Harmonic Numbers, Bernoulli Numbers, Fibonacci Numbers . . . 658
Appendix B — Index to Notations . . . . . . . . . . . . . . . . . . 660
Appendix C — Index to Algorithms and Theorems . . . . . . . . . . 666
Appendix D — Index to Combinatorial Problems . . . . . . . . . . . 667
Appendix E — Answers to Puzzles in the Answers . . . . . . . . . . 671
Index and Glossary . . . . . . . . . . . . . . . . . . . . . . . . . 674

We — or the Black Chamber — have a little agreement with [Knuth];

he doesn’t publish the real Volume 4 of The Art of Computer Programming,
and they don’t render him metabolically challenged.
— CHARLES STROSS, The Atrocity Archive (2001)
In books of this nature I can only suggest you keep it
as simple as the subject will allow.
— KODE VICIOUS (2012)

-17
-18
 MATHEMATICAL PRELIMINARIES REDUX
Many parts of this book deal with discrete probabilities, namely with a finite or
countably infinite set Ω of atomic events ω, each of which has a given probability
Pr(ω), where
0 ≤ Pr(ω) ≤ 1 and Pr(ω) = 1. (1)
ω∈Ω
This set Ω, together with the function Pr, is called a “probability space.” For
example, Ω might be the set of all ways to shuffle a pack of 52 playing cards,
with Pr(ω) = 1/52! for every such arrangement.
An event is, intuitively, a proposition that can be either true or false with
certain probability. It might, for instance, be the statement “the top card is an
ace,” with probability 1/13. Formally, an event A is a subset of Ω, namely the
set of all atomic events for which the corresponding proposition A is true; and
Pr(A) = Pr(ω) = Pr(ω)[ ω ∈ A ]. (2)
ω∈A ω∈Ω
A random variable is a function that assigns a value to every atomic event.
We typically use uppercase letters for random variables, and lowercase letters

for the values that they might assume; thus, we might say that the probability
of the event X = x is Pr(X = x) = ω∈Ω Pr(ω)[ X (ω) = x ]. In our playing card
example, the top card T is a random variable, and we have Pr(T = Q ♠) = 1/52.
(Sometimes, as here, the lowercase-letter convention is ignored.)
The random variables X1 , . . . , Xk are said to be independent if
Pr(X1 = x1 and · · · and Xk = xk ) = Pr(X1 = x1 ) . . . Pr(Xk = xk ) (3)
for all (x1 , . . . , xk ). For example, if F and S denote the face value and suit of
the top card T , clearly F and S are independent. Hence in particular we have
Pr(T = Q ♠) = Pr(F = Q) Pr(S = ♠). But T is not independent of the bottom
card, B ; indeed, we have Pr(T = t and B = b) = 1/522 for any cards t and b.
A system of n random variables is called k-wise independent if no k of
its variables are dependent. With pairwise (2-wise) independence, for example,
we could have variable X independent of Y, variable Y independent of Z , and
variable Z independent of X; yet all three variables needn’t be independent
(see exercise 6). Similarly, k-wise independence does not imply (k + 1)-wise
independence. But (k + 1)-wise independence does imply k-wise independence.
The conditional probability of an event A, given an event B , is
A ∩ B ) = Pr(A and B ) ,
Pr(A | B ) = Pr(Pr( (4)
B) Pr(B )
1

1
2 MATHEMATICAL PRELIMINARIES REDUX
when Pr(B ) > 0, otherwise it’s Pr(A). Imagine breaking the whole space Ω into
two parts, Ω = B and Ω = B = Ω \ B , with Pr(Ω ) = Pr(B ) and Pr(Ω ) =
1 − Pr(B ). If 0 < Pr(B ) < 1, and if we assign new probabilities by the rules
Pr  (ω) = Pr(ω | Ω ) = Pr(ωPr(Ω)[ ω ∈ Ω ] , Pr  (ω) = Pr(ω | Ω ) = Pr(ω)[ ω ∈ Ω ] ,
) Pr(Ω )
we obtain new probability spaces Ω and Ω , allowing us to contemplate a world
where B is always true and another world where B is always false. It’s like taking
two branches in a tree, each of which has its own logic. Conditional probability is
important for the analysis of algorithms because algorithms often get into differ-
ent states where different probabilities are relevant. Notice that we always have
Pr(A) = Pr(A | B ) · Pr(B ) + Pr(A | B ) · Pr(B ). (5)
The events A1 , . . . , Ak are said to be independent if the random variables
[ A1 ], . . . , [ Ak ] are independent. (Bracket notation applies in the usual way to
events-as-statements, not just to events-as-subsets: [ A ] = 1 if A is true, otherwise

Pr
 
[ A ] = 0.) Exercise 20 proves that this happens if and only if
Aj = Pr(Aj ), for all J ⊆ {1, . . . , k}. (6)
j ∈J j ∈J
In particular, events A and B are independent if and only if Pr(A | B ) = Pr(A).
When the values of a random variable X are real numbers or complex
numbers, we’ve defined its expected value E X in Section 1.2.10: We said that
EX = X (ω) Pr(ω) = x Pr(X = x), (7)
ω∈Ω x
provided that this definition makes sense when the sums are taken over infinitely
many nonzero values. (The sum should be absolutely convergent.) A simple but
extremely important case arises when A is any event, and when X = [ A ] is a
binary random variable representing the truth of that event; then
E[ A ] = [ A ](ω) Pr(ω) = [ ω ∈ A ] Pr(ω) = Pr(ω) = Pr(A). (8)
ω∈Ω ω∈Ω ω∈A
We’ve also noted that the expectation of a sum, E(X1 + · · · + Xk ), always
equals the sum of the expectations, (E X1 ) + · · · + (E Xk ), whether or not the
random variables Xj are independent. Furthermore the expectation of a product,
E X1 . . . Xk , is the product of the expectations, (E X1 ) . . . (E Xk ), if those vari-
 
ables do happen to be independent. In Section 3.3.2 we defined the covariance,
covar(X, Y ) = E (X − E X )(Y − E Y ) = (E XY ) − (E X )(E Y ), (9)
which tends to measure the way X and Y depend on each other. The variance,
var(X ), is covar(X, X ); the middle formula in (9) shows why it is nonnegative
whenever the random variable X takes on only real values.
All of these notions of expected value carry over to conditional expectation,
E(X | A) = Pr(ω) =
X (ω) Pr( x Pr(X Pr(
= x and A) , (10)
ω∈A A ) x A)

2
 MATHEMATICAL PRELIMINARIES REDUX 3
conditioned on any event A, when we want to work in the probability space for
which A is true. (If Pr(A) = 0, we define E(X | A) = E X .) One of the most
important formulas, analogous to (5), is
EX = E(X | Y = y) Pr(Y = y)
y
= x Pr(X = x | Y = y) Pr(Y = y). (11)
y x
Furthermore there’s also another important kind of conditional expectation:
When X and Y are random variables, it’s often helpful to write ‘E(X | Y )’ for
 
“the expectation of X given Y .” Using that notation, Eq. (11) becomes simply
E X = E E(X | Y ) . (12)
This is a truly marvelous identity, great for hand-waving and for impressing
outsiders — except that it can be confusing until you understand what it means.
In the first place, if Y is a Boolean variable, ‘E(X | Y )’ might look as if it
means ‘E(X | Y =1)’, thus asserting that Y is true, just as ‘E(X | A)’ asserts the
truth of A in (10). No; that interpretation is wrong, quite wrong. Be warned.
In the second place, you might think of E(X | Y ) as a function of Y . Well,
yes; but the best way to understand E(X | Y ) is to regard it as a random variable.
That’s why we’re allowed to compute its expected value in (12).
All random variables are functions of the atomic events ω. The value of
E(X | Y ) at ω is the average of X (ω ) over all events ω such that Y (ω ) = Y (ω):
E(X | Y )(ω) = X (ω ) Pr(ω )[ Y (ω ) = Y (ω)]/ Pr(Y = Y (ω)). (13)
ω ∈Ω
Similarly, E(X | Y1 , . . . , Yr ) averages over events with Yj (ω )= Yj (ω) for 1 ≤ j ≤ r.
For example, suppose X1 through Xn are binary random variables con-

strained by the condition that ν (X1 . . . Xn ) = X1 + · · · + Xn = m, where m and n
are constants with 0 ≤ m ≤ n; all mn such bit vectors X1 . . . Xn are assumed to
be equally likely. Clearly E X1 = m/n. But what is E(X2 | X1 )? If X1 = 0, the
expectation of X2 is m/(n − 1); otherwise that expectation is (m − 1)/(n − 1);
consequently E(X2 | X1 ) = (m − X1 )/(n − 1). And what is E(Xk | X1 , . . . , Xk−1 )?
The answer is easy, once you get used to the notation: If ν (X1 . . . Xk−1 ) = r,
then Xk . . . Xn is a random bit vector with ν (Xk . . . Xn ) = m − r; hence the
average value of Xk will be (m − r)/(n + 1 − k) in that case. We conclude that
E(Xk | X1 , . . . , Xk−1 ) = m − νn(X 1 . . . Xk−1 ) , for 1 ≤ k ≤ n.
+1−k (14)
The random variables on both sides of these equations are the same.
Inequalities. In practice we often want to prove that certain events are rare,
in the sense that they occur with very small probability. Conversely, our goal
is sometimes to show that an event is not rare. And we’re in luck, because
mathematicians have devised several fairly easy ways to derive upper bounds or
lower bounds on probabilities, even when the exact values are unknown.

3
4 MATHEMATICAL PRELIMINARIES REDUX
We’ve already discussed the most important technique of this kind in Sec-
tion 1.2.10. Stated in highly general terms, the basic idea can be formulated as
follows: Let f be any nonnegative function such that f (x) ≥ s > 0 when x ∈ S .
Then
Pr(X ∈ S ) ≤ E f (X )/s, (15)
provided that Pr(X ∈ S ) and E f (X ) both exist. For example, f (x) = |x| yields
Pr(|X | ≥ m) ≤ E |X | /m (16)
whenever m > 0. The proof is amazingly simple, because we obviously have
E f (X ) ≥ Pr(X ∈ S ) · s + Pr(X ∈/ S ) · 0 . (17)
Formula (15) is often called Markov’s inequality, because A. A. Markov discussed
the special case f (x) = |x|a in Izvıestı̄ıa Imp. Akad. Nauk (6) 1 (1907), 707–716.
If we set f (x) = (x − E X )2 , we get the famous 19th-century inequality of
Bienaymé and Chebyshev:
 
Pr |X − E X | ≥ r ≤ var(X )/r2 . (18)
The case f (x) = eax is also extremely useful.
Another fundamental estimate, known as Jensen’s inequality [Acta Mathe-
matica 30 (1906), 175–193], applies to convex functions f ; we’ve seen it so far
only as a “hint” to exercise 6.2.2–36(!). The real-valued function f is said to be
convex in an interval I of the real line, and −f is said to be concave in I , if
f (px + qy) ≤ pf (x) + qf (y) for all x, y ∈ I, (19)
whenever p ≥ 0, q ≥ 0, and p+q = 1. This condition turns out to be equivalent to
saying that f  (x) ≥ 0 for all x ∈ I , if f has a second derivative f  . For example,
the functions eax and x2n are convex for all constants a and all nonnegative
integers n; and if we restrict consideration to positive values of x, then f (x) = xn
is convex for all integers n (notably f (x) = 1/x when n = −1). The functions
ln(1/x) and x ln x are also convex for x > 0. Jensen’s inequality states that
f (E X ) ≤ E f (X ) (20)
when f is convex in the interval I and the random variable X takes values only
in I . (See exercise 42 for a proof.) For example, we have 1/ E X ≤ E(1/X ) and
ln E X ≥ E ln X and (E X ) ln E X ≤ E(X ln X ), when X is positive, since the
function ln x is concave for x > 0. Notice that (20) actually reduces to the very
definition of convexity, (19), in the special case when X = x with probability p
and X = y with probability q = 1 − p.
Next on our list of remarkably useful inequalities are two classical results
that apply to any random variable X whose values are nonnegative integers:
Pr(X > 0) ≤ E X ; (“the first moment principle”) (21)
Pr(X > 0) ≥ (E X ) / (E X ) . (“the second moment principle”) (22)
2 2
Formula (21) is obvious, because the left side is p1 + p2 + p3 + · · · when pk is the
probability that X = k, while the right side is p1 + 2p2 + 3p3 + · · · .

4
 MATHEMATICAL PRELIMINARIES REDUX 5
Formula (22) isn’t quite so obvious; it is p1 + p2 + p3 + · · · on the left and
(p1 + 2p2 + 3p3 + · · · )2/(p1 + 4p2 + 9p3 + · · · ) on the right. However, as we saw
with Markov’s inequality, there is a remarkably simple proof, once we happen to
discover it: If X is nonnegative but not always zero, we have
E X 2 = E(X 2 | X > 0) Pr(X > 0) + E(X 2 | X = 0) Pr(X = 0)
 
= E(X 2 | X > 0) Pr(X > 0)
≥ E(X | X > 0) 2 Pr(X > 0) = (E X )2 / Pr(X > 0). (23)
In fact this proof shows that the second moment principle is valid even when X is
not restricted to integer values (see exercise 46). Furthermore the argument can
be strengthened to show that (22) holds even when X can take arbitrary negative
values, provided only that E X ≥ 0 (see exercise 47). See also exercise 118.
Exercise 54 applies (21) and (22) to the study of random graphs.
Another important inequality, which applies in the special case where X =
X1 + · · · + Xm is the sum of binary random variables Xj , was introduced more
recently by S. M. Ross [Probability, Statistics, and Optimization (New York:
Wiley, 1994), 185–190], who calls it the “conditional expectation inequality”:
m E Xj
Pr(X > 0) ≥ . (24)
j =1 E(X | Xj =1)
Ross showed that the right-hand side of this inequality is always at least as big
as the bound (E X )2/(E X 2 ) that we get from the second moment principle (see
exercise 50). Furthermore, (24) is often easier to compute, even though it may
look more complicated at first glance.
For example, his method applies nicely to the problem of estimating a
reliability polynomial, f (p1 , . . . , pn ), when f is a monotone Boolean function;
here pj represents the probability that component j of a system is “up.” We ob-
served in Section 7.1.4 that reliability polynomials can be evaluated exactly, using
BDD methods, when n is reasonably small; but approximations are necessary
when f gets complicated. The simple example f (x1 , . . . , x5 ) = x1 x2 x3 ∨ x2 x3 x4 ∨
x4 x5 illustrates Ross’s general method: Let (Y1 , . . . , Y5 ) be independent binary
random variables, with E Yj = pj ; and let X = X1 +X2 +X3 , where X1 = Y1 Y2 Y3 ,
X2 = Y2 Y3 Y4 , and X3 = Y4 Y5 correspond to the prime implicants of f . Then
Pr(X > 0) = Pr(f (Y1 , . . . , Y5 ) = 1) = E f (Y1 , . . . , Y5 ) = f (p1 , . . . , p5 ), because
the Y ’s are independent. And we can evaluate the bound in (24) easily:
Pr(X > 0) ≥ 1 +pp1 p+ 2 p3 p2 p3 p4 +
p p p + 1 + p p p p + p p + 1.
+ p4 p5 (25)
4 4 5 1 5 1 2 3 2 3
If, for example, each pj is 0.9, this formula gives ≈ 0.848, while (E X )2/(E X 2 ) ≈
0.847; the true value, p1 p2 p3 + p2 p3 p4 + p4 p5 − p1 p2 p3 p4 − p2 p3 p4 p5 , is 0.9558.
Many other important inequalities relating to expected values have been
discovered, of which the most significant for our purposes in this book is the
FKG inequality discussed in exercise 61. It yields easy proofs that certain events
are correlated, as illustrated in exercise 62.

5
6 MATHEMATICAL PRELIMINARIES REDUX
Martingales. A sequence of dependent random variables can be difficult to
analyze, but if those variables obey invariant constraints we can often exploit
their structure. In particular, the “martingale” property, named after a classic
betting strategy (see exercise 67), proves to be amazingly useful when it applies.
Joseph L. Doob featured martingales in his pioneering book Stochastic Processes
(New York: Wiley, 1953), and developed their extensive theory.
The sequence Zn = Z0 , Z1 , Z2 , . . . of real-valued random variables is
called a martingale if it satisfies the condition
E(Zn+1 | Z0 , . . . , Zn ) = Zn for all n ≥ 0. (26)
(We also implicitly assume, as usual, that the expectations E Zn are well defined.)
For example, when n = 0, the random variable E(Z1 | Z0 ) must be the same as
the random variable Z0 (see exercise 63).
Figure P illustrates George Pólya’s famous “urn model” [F. Eggenberger
and G. Pólya, Zeitschrift für angewandte Math. und Mech. 3 (1923), 279–289],
which is associated with a particularly interesting martingale. Imagine an urn
that initially contains two balls, one red and one black. Repeatedly remove a
randomly chosen ball from the urn, then replace it and contribute a new ball of
the same color. The numbers (r, b) of red and black balls will follow a path in
the diagram, with the respective local probabilities indicated on each branch.
One can show without difficulty that all n +1 nodes on level n of Fig. P will
be reached with the same probability, 1/(n + 1). Furthermore, the probability
that a red ball is chosen when going from any level to the next is always 1/2. Thus
the urn scheme might seem at first glance to be rather tame and uniform. But
in fact the process turns out to be full of surprises, because any inequity between
red and black tends to perpetuate itself. For example, if the first ball chosen is
black, so that we go from (1, 1) to (1, 2), the probability is only 2 ln 2 − 1 ≈ .386
that the red balls will ever overtake the black ones in the future (see exercise 88).
One good way to analyze Pólya’s process is to use the fact that the ratios
r/(r + b) form a martingale. Each visit to the urn changes this ratio either to
(r +1)/(r + b +1) (with probability r/(r + b)) or to r/(r + b +1) (with probability
b/(r + b)); so the expected new ratio is (rb + r2 + r)/((r + b)(r + b +1)) = r/(r + b),
no different from what it was before. More formally, let X0 = 1, and for n > 0
let Xn be the random variable ‘[the nth ball chosen is red]’. Then there are
X0 + · · · + Xn red balls and X 0 + · · · + X n + 1 black balls at level n of Fig. P;
and the sequence Zn is a martingale if we define
Zn = (X0 + · · · + Xn )/(n + 2). (27)
In practice it’s usually most convenient to define martingales Z0 , Z1 , . . .
in terms of auxiliary random variables X0 , X1 , . . . , as we’ve just done. The
sequence Zn is said to be a martingale with respect to the sequence Xn if
Zn is a function of (X0 , . . . , Xn ) that satisfies
E(Zn+1 | X0 , . . . , Xn ) = Zn for all n ≥ 0. (28)

6
 MATHEMATICAL PRELIMINARIES REDUX 7
Level 0 11
,

1/2 1/2
Level 1 12
, 21
,

2/3 1/3 1/3 2/3

Level 2 13
, 22
, 31
,

3/4 1/4 2/4 2/4 1/4 3/4

Level 3 14
, 23
, 32 , 41
,

4/5 1/5 3/5 2/5 2/5 3/5 1/5 4/5

Fig. P. Pólya’s urn model. The probability of taking any downward path
from (1, 1) to (r, b) is the product of the probabilities shown on the branches.

Furthermore we say that a sequence Yn is fair with respect to the sequence Xn

if Yn is a function of (X0 , . . . , Xn ) that satisfies the simpler condition
E(Yn+1 | X0 , . . . , Xn ) = 0 for all n ≥ 0; (29)
and we call Yn fair whenever
E(Yn+1 | Y0 , . . . , Yn ) = 0 for all n ≥ 0. (30)
Exercise 77 proves that (28) implies (26) and that (29) implies (30); thus an
auxiliary sequence Xn is sufficient but not necessary for defining martingales
and fair sequences.
Whenever Zn is a martingale, we obtain a fair sequence Yn by letting
Y0 = Z0 and Yn = Zn − Zn−1 for n > 0, because the identity E(Yn+1 |
Z0 , . . . , Zn ) = E(Zn+1 − Zn | Z0 , . . . , Zn ) = Zn − Zn shows that Yn is fair
with respect to Zn . Conversely, whenever Yn is fair, we obtain a martingale
Zn by letting Zn = Y0 + · · · + Yn , because the identity E(Zn+1 | Y0 , . . . , Yn ) =
E(Zn + Yn+1 | Y0 , . . . , Yn ) = Zn shows that Zn is a martingale with respect
to Yn . In other words, fairness and martingaleness are essentially equivalent.
The Y ’s represent unbiased “tweaks” that change one Z to its successor.
It’s easy to construct fair sequences. For example, every sequence of inde-
pendent random variables with mean 0 is fair. And if Yn is fair with re-
spect to Xn , so is the sequence Yn defined by Yn = fn (X0 , . . . , Xn−1 )Yn
when fn (X0 , . . . , Xn−1 ) is almost any function whatsoever! (We need only
keep fn small enough that E Yn is well defined.) In particular, we can let
fn (X0 , . . . , Xn−1) = 0 for all large n, thereby making Zn eventually fixed.
A sequence of functions Nn (x0 , . . . , xn−1 ) is called a stopping rule if each
value is either 0 or 1 and if Nn (x0 , . . . , xn−1 ) = 0 implies Nn+1 (x0 , . . . , xn ) = 0.
We can assume that N0 = 1. The number of steps before stopping, with respect
to a sequence of random variables Xn , is then the random variable
N = N1 (X0 ) + N2 (X0 , X1 ) + N3 (X0 , X1 , X2 ) + · · · . (31)
(Intuitively, Nn (x0 , . . . , xn−1 ) means [the values X0 = x0 , . . . , Xn−1 = xn−1 do
not stop the process]; hence it’s really more about “going” than “stopping.”)
Any martingale Zn = Y0 + · · · + Yn with respect to Xn can be adapted to

7
8 MATHEMATICAL PRELIMINARIES REDUX
stop with this strategy if we change it to Zn = Y0 + · · · + Yn , where Yn =
Nn (X0 , . . . , Xn−1 )Yn . Gamblers who wish to “quit when ahead” are using the
stopping rule Nn+1 (X0 , . . . , Xn ) = [ Zn ≤ 0], when Zn is their current balance.
Notice that if the stopping rule always stops after at most m steps — in
other words, if the function Nm (x0 , . . . , xm−1 ) is identically zero — then we have
Zm = ZN , because Zn doesn’t change after the process has stopped. Therefore
E ZN = E Zm = E Z0 = E Z0 : No stopping rule can change the expected outcome
of a martingale when the number of steps is bounded.
An amusing game of chance called Ace Now illustrates this optional stopping
principle. Take a deck of cards, shuffle it and place the cards face down; then
turn them face up one at a time as follows: Just before seeing the nth card, you
are supposed to say either “Stop” or “Deal,” based on the cards you’ve already
observed. (If n = 52 you must say “Stop.”) After you’ve decided to stop, you
win $12 if the next card is an ace; otherwise you lose $1. What is the best
strategy for playing this game? Should you hold back until you have a pretty
good chance at the $12? What is the worst strategy? Exercise 82 has the answer.
Tail inequalities from martingales. The essence of martingales is equality
of expectations. Yet martingales turn out to be important in the analysis of
algorithms because we can use them to derive inequalities, namely to show that
certain events occur with very small probability.
To begin our study, let’s introduce inequality into Eq. (26): A sequence Zn
is called a submartingale if it satisfies
E(Zn+1 | Z0 , . . . , Zn ) ≥ Zn for all n ≥ 0. (32)
Similarly, it’s called a supermartingale if ‘≥’ is changed to ‘≤’ in the left-hand
part of this definition. (Thus a martingale is both sub- and super-.) In a
submartingale we have E Z0 ≤ E Z1 ≤ E Z2 ≤ · · · , by taking expectations in (32).
A supermartingale, similarly, has ever smaller expectations as n grows. One way
to remember the difference between submartingales and supermartingales is to
observe that their names are the reverse of what you might expect.
Submartingales are significant largely because of the fact that they’re quite
common. Indeed, if Zn is any martingale and if f is any convex function, then
f (Zn ) is a submartingale (see exercise 84). For example, the sequences |Zn |
and max(Zn , c) and Zn2 and eZn all are submartingales whenever Zn is
known to be a martingale. If, furthermore, Zn is always positive, then Zn3 and
1/Zn and ln(1/Zn ) and Zn ln Zn , etc., are submartingales.
If we modify a submartingale by applying a stopping rule, it’s easy to see that
we get another submartingale. Furthermore, if that stopping rule is guaranteed
to quit within m steps, we’ll have E Zm ≥ E ZN = E ZN = E Zm . Therefore no
stopping rule can increase the expected outcome of a submartingale, when the
number of steps is bounded.
That comparatively simple observation has many important consequences.
For example, exercise 86 uses it to give a simple proof of the so-called “maximal

8
 MATHEMATICAL PRELIMINARIES REDUX 9

 
inequality”: If Zn is a nonnegative submartingale then
Pr max(Z0 , Z1 , . . . , Zn ) ≥ x ≤ E Zn /x, for all x > 0. (33)
   
Special cases of this inequality are legion. For instance, martingales Zn satisfy
 2 2 
Pr max |Z0 |, |Z1 |, . . . , |Zn | ≥ x ≤ E |Zn |/x,
Pr max(Z0 , Z1 , . . . , Zn ) ≥ x ≤ E Zn /x,
2 2
for all x > 0;
for all x > 0.
(34)
(35)
Relation (35) is known as Kolmogorov’s inequality, because A. N. Kolmogorov
proved it when Zn = X1 + · · · + Xn is the sum of independent random variables
with E Xk = 0 and var Xk = σk2 for 1 ≤ k ≤ n [Math. Annalen 99 (1928), 309–
 
311]. In that case var Zn = σ12 + · · · + σn2 = σ2 , and the inequality can be written
 
Pr |X1 | < tσ, |X1 + X2 | < tσ, . . . , |X1 + · · · + Xn | < tσ ≥ 1 − 1/t2 . (36)
Chebyshev’s inequality gives only Pr |X1 + · · · + Xn | < tσ ≥ 1 − 1/t2 , which is
a considerably weaker result.
Another important inequality applies in the common case where we have
good bounds on the terms Y1 , . . . , Yn that enter into the standard representation
Zn = Y0 + Y1 + · · · + Yn of a martingale. This one is called the Hoeffding–Azuma
inequality, after papers by W. Hoeffding [J. Amer. Statistical Association 58
(1963), 13–30] and K. Azuma [Tôhoku Math. Journal (2) 19 (1967), 357–367].
It reads as follows: If Yn is any fair sequence with an ≤ Yn ≤ bn when Y0 , Y1 ,
. . . , Yn−1 are given, then
Pr(Y1 + · · · + Yn ≥ x) ≤ e−2x /((b1 −a1 ) +···+(bn −an ) ) . (37)
2 2 2

The same bound applies to Pr(Y1 + · · · + Yn ≤ −x), since −bn ≤ −Yn ≤ −an ; so
Pr(| Y1 + · · · + Yn | ≥ x) ≤ 2e−2x /((b1 −a1 ) +···+(bn −an ) ) . (38)
2 2 2

Exercise 90 breaks the proof of this result into small steps. In fact, the proof
even shows that an and bn may be functions of {Y0 , . . . , Yn−1 }.
Applications. The Hoeffding–Azuma inequality is useful in the analysis of
many algorithms because it applies to “Doob martingales,” a very general class
of martingales that J. L. Doob featured as Example 1 in his Stochastic Processes
(1953), page 92. (In fact, he had already considered them many years earlier,
in Trans. Amer. Math. Soc. 47 (1940), 486.) Doob martingales arise from any
sequence of random variables Xn , independent or not, and from any other
random variable Q: We simply define
Zn = E(Q | X0 , . . . , Xn ). (39)
Then, as Doob pointed out, the resulting sequence is a martingale (see exercise
91). In our applications, Q is an aspect of some algorithm that we wish to study,
and the variables X0 , X1 , . . . reflect the inputs to the algorithm. For example,
in an algorithm that uses random bits, the X ’s are those bits.
Consider a hashing algorithm in which t objects are placed into m random
lists, where the nth object goes into list Xn ; thus 1 ≤ Xn ≤ m for 1 ≤ n ≤ t, and
we assume that each of the mt possibilities is equally likely. Let Q(x1 , . . . , xt ) be

9
Other documents randomly have
different content
would soon be far behind, and June and he in a strange country and
a new life would begin their dream of love.
 CHAPTER VII
THE COLONEL COMES BACK
Jerry’s plans had been laid with the utmost secrecy and care. It
behooved him to be wary, for he knew that detection would mean
death. Neither the Colonel nor Black Dan would have hesitated to
shoot him like a dog if they had known what he contemplated, and
working day by day in an office with these men, in a town the
smallness and isolation of which rendered every human figure a
segregated and important unit, it required all the shrewdness of
which he was master to mature his design and arouse no suspicion.
The time had now come when everything was suddenly propitious.
Had the Prince of Darkness been giving Jerry’s affairs his particular
attention, circumstances could not have fallen together more
conveniently for the furthering of his purpose.
In the office of the Cresta Plata it was arranged that every two weeks
he should be given three or four days off to go to San Francisco and
visit his wife. These holidays, which were grudgingly doled out by
Black Dan, always included the Sunday, as the older man was
determined his son-in-law should have as little immunity from work
as possible. In the middle of the week Jerry was informed that he
could leave for San Francisco on the following Saturday morning to
report again at the office on Wednesday.
The granting of this five days’ leave of absence made the elopement
easy of accomplishment, robbing it of the danger of detection that
Jerry realized and shrank from. He and June could leave on Friday
night and take the overland train eastward. They would have five
days’ start before discovery was made, and in five days they would
be so far on their journey that it would be easy for them to conceal
themselves in some of the larger towns along the route. Mercedes,
who was a bad correspondent, could be trusted not to write to her
father, and Allen, according to June’s artless revelations, was gone
for a much longer time than he wanted known. Finally the last and
most serious obstacle was removed in the shape of the Colonel.
Jerry being in the office knew that his enemy would not be back
before Tuesday or Wednesday, as the work of inspecting the pumps
had been slower than was anticipated.
Months of waiting and planning could not have arranged matters
more satisfactorily. Luck, once again, was on his side, as it had been
so often in the past.
Early on the Friday morning he went to the livery stable that he
always patronized, and where he knew the finest team of roadsters
in Nevada was for hire. Mining men of that day were particular about
their horses. There were animals in the Virginia stables whose
superiors could not be found west of New York. The especial pair
that Jerry wanted were only leased to certain patrons of the stable,
but Jerry, an expert on horseflesh, besides being Black Dan
Gracey’s son-in-law, had no difficulty in securing them for that
evening.
He had had some idea of driving into Reno himself and letting June
come in on the train, but he had a fear that, left alone, she might
weaken. To be sure of her he must be with her. Moreover, there was
little risk in driving in together. They would not start until after dark
and their place of rendezvous would be a ruined cabin some
distance beyond the Utah hoisting works on the Geiger grade. The
spot would be deserted at that hour, and even if it were not, the
spectacle of a buggy and pair of horses was so common that it
would be taken for that of some overworked superintendent driving
into Reno on a sudden business call.
From the stable he returned to the office and alone there wrote a
hasty letter to June. He had told her the outline of his plan, and that
Friday would be the day, but he had given her no details of what their
movements would be. Now he wrote telling her minutely of the time
and place of departure and impressing upon her not to be late. He
would, of course, be there before her, waiting in the buggy. There
was a party of Eastern visitors to be taken over the mine in the
afternoon, and it would be easy for him to get away from them,
leaving them with Marsden, the foreman, change his clothes, and be
at the place indicated before she was. He was still fearful that she
might fail him. Now, as the hour approached, he was so haunted by
the thought that he asked her to send at least a few words of answer
by his messenger.
Half an hour later Black Dan entered the office and paused by his
son-in-law’s desk to give him some instructions as to the Eastern
visitors and the parts of the mine they were to be shown. They were
people of importance from New York, the men being heavy
shareholders in the Cresta Plata and the Con. Virginia. The ladies of
the party were to be relegated to the care of Jerry and Marsden the
foreman, and not to be taken below the thousand-foot level. Black
Dan and Barney Sullivan would take the men farther down. Jerry
was to be ready in the hoisting works at four o’clock.
As Black Dan was concluding his instructions Jerry’s messenger
reëntered the office and handed the young man a small, pale gray
envelope. It was obviously a feminine communication, and its
recipient, under the darkly scrutinizing eye of his father-in-law,
flushed slightly, but he gave no other sign of consciousness, and as
Black Dan passed on to the inner office, he sat down and opened
the letter.
It was only a few lines in June’s delicate hand-writing:
“I will be there. I go to my ruin, Jerry, for you. Will there be anything
in our life together that will make me forget that? June.”
Jerry read it over several times. It certainly did not breathe an
exalted gladness. Away from him she always seemed in this
condition of fear and doubt. It was his presence, his hand upon her,
that made her tremulously, submissively his. He would not be sure of
her till she was beside him to-night in the buggy.
Would that hour ever come? He looked at the clock ticking on the
wall. With every passing moment his exaltation seemed to grow
stronger. It was difficult for him to be quiet, not to stop and talk to
everybody that he encountered with a feverish loquacity. The slowly
gathering pressure of the last month seemed to culminate on this
day of mad rebellion. Within the past two weeks his stocks had
increased largely in value. He was a rich man, and to-night with the
woman he loved beside him he would be free. He and she, free in
the great outside world, free to love and to live as they would. Would
the day never pass and the night never come?
In San Francisco the Colonel was completing the business of the
pumps as quickly as he could. He felt that he was getting to be a
foolish old man, but he could not shake off his worry about June. Her
words and appearance at their last interview kept recurring to him.
Many times in the past year he had seen her looking pitifully fragile
and known her to be unhappy, but he had never before felt the
poignant anxiety about her that he now experienced.
Despite his desire to get back with as much speed as possible,
unforeseen delays occurred, and instead of returning on Friday, as
he had hoped, he saw that he would not be back before Wednesday.
He wrote this to June in a letter full of the anxious solicitude he felt.
To this he received no answer, and, his worry increasing, he was
about to telegraph her when he received a piece of information that
swept all minor matters from his mind.
On Thursday at midday he was lunching with a friend at the club,
when, in the course of conversation, his companion asked him if he
knew the whereabouts of Beauregard Allen. The words were
accompanied with a searchingly significant look. The Colonel,
answering that Allen was in Virginia, paused in his meal and became
quietly attentive. He knew more than others of Allen’s situation. Of
late he had scented catastrophe ahead of his one-time comrade.
The man’s face opposite him struck an arrow of suspicion through
his mind. He put down his wine glass and sat listening, his
expression one of frowning concentration.
His friend was a merchant with a large shipping business between
San Francisco and Australia. That morning he had been to the docks
to see a ship about to sail, which carried a cargo of his own. The
ship took few passengers, only two or three, he thought. While
conversing with the captain he had seen distinctly in the doorway of
an open cabin Beauregard Allen unpacking a valise. In answer to his
question the captain had said it was one of his passengers taken on
that morning. He had brought no trunks, only two valises, and given
his name as John Montgomery.
“It was Beauregard Allen,” the Colonel’s informant continued. “The
man’s no friend of mine, but I’ve seen him round here for years. He
looked up and saw me and drew back quick, as if he did not want to
be recognized.”
“He’d taken passage this morning, you say?” asked the Colonel.
“Yes, to Melbourne. There was only one other passenger, a drunken
boy being sent on a long sea voyage by his parents. They’ll make a
nice, interesting pair.”
The Colonel looked at his plate silently. He was sending his thoughts
back over the last year, trying to collect data that might throw some
light on what he had just heard.
“You’re certain it was Allen, not a chance likeness?” he said slowly.
“I’ll take my oath of it. Why, I’ve seen the man for the past four years
dangling around here. I know his face as well as I know yours, and I
had a good look at it before he saw me and jumped back. He’s got in
too deep and skipped. Everybody has been wondering how he kept
on his feet so long.”
“He’s in pretty deep, sure enough,” said the Colonel absently. “You
said Melbourne was the port? When do they sail?”
“Midday to-day. They’re off by now. They’ll be outside the heads
already with this breeze.”
The Colonel asked a few more questions and then rose and excused
himself. His business was pressing.
His first action was to send a telegram to Rion Gracey, asking him if
Allen had left Virginia and where June was. The answer was to be
sent to the club. Then he went forth. His intention was to inquire at
the hotels patronized by Allen on his frequent visits to the city. As he
went from place to place the conviction that the man seen by his
friend had been June’s father, and that he had fled, strengthened
with every moment.
A feverish anxiety about June took possession of him. If her father
had decamped leaving her alone, she would have to face his angry
creditors. He thought of her as he had last seen her, exposed to
such an experience, and his heart swelled with pity and rage.
Possibly she knew, had guessed what was coming and had begged
him to stay with her to protect and care for her in a position for which
she was so little fitted. And he had left her—left her to face it alone!
He returned to the club, having heard no word of Allen, and found
Rion’s answer to his telegram. It ran:
“Allen left for coast Wednesday morning. June here. What’s amiss?”
It seemed to the Colonel complete confirmation of his fears. Allen
leaving Wednesday morning would reach San Francisco some time
that night. Evidently his plans had been made beforehand, for the
ship he had taken was one of the fastest merchantmen on the
Pacific and was scheduled to leave at midday Thursday.
Nothing was suspected at Virginia yet, and June was there alone. At
any moment now, the information being in the hands of more than
one person, Allen’s flight might be made public, and she, his only
representative, would become the victim of the rage of the petty
creditors who would swarm about her. He was the one human being
upon whom she could call. No duty or business would hold him from
her. A thrill of something like joy passed through him when he
realized that now, at last, he could stand between her and all trouble
—a lion with its cub behind it.
He took the evening train for Virginia, hoping to reach Reno the next
morning and catch the branch line into the mining town. But luck was
against him. A snow-shed was down near the summit. Though it was
only the latter half of September, a premature blizzard wrapped the
mountain heights in a white mist. For eight hours the train lay
blocked on an exposed ridge, and it was late afternoon when it finally
set the Colonel down at Reno.
The delays had only accelerated his desire to be with June. During
the long hours of waiting his imagination had been active, picturing
her in various distressing positions, besieged by importunate
creditors. He hired the fastest saddle horse in the Reno stables and
rode the twenty-one miles into Virginia in an hour. It was dark when
he reached there. The swift ride through the sharp autumnal air had
braced his nerves. He was as anxious as ever to see her, but he
thought that before he did so he would stop for a few moments at the
Cresta Plata and see Rion, explain his early return, and learn if
anything was known in Virginia of Allen’s flight.
The office was already lighted up and behind it the great bulk of
hoisting works loomed into the night, its walls cut with the squares of
illumined windows, its chimneys rising black and towering against
the stars. A man who came forward to take his horse told him that
the gentlemen were all in the mine with a party of visitors. The
Colonel, hearing this, turned his steps from the office to the door of
the hoisting works a few yards beyond.
The building, full of shadows despite the lanterns and gas jets
ranged along its walls, looked vacant and enormous in its lofty
spaciousness. The noise of machinery echoed through it, the
vibration shaking it as if it were a shell built about the intricacies of
wheels, bands, and sheaves that whirred and slid in complicated,
humming swiftness against the ceiling. The light struck gleams from
the car tracks that radiated from the black hole of the shaft mouth
where it opened in the middle of the floor. It was divided into four
compartments, and from these a thin column of steam arose and
floated up to the roof. Here and there a few men were moving about,
and aloft behind their engines were the four engineers. They were
mute as statues, their eyes fixed on the dials in front of them which
registered the movements of the cages underground; their ears on
the alert to catch the notes of their bells, to them intelligible as the
words of a spoken language. Near the shaft mouth sitting on an
overturned box was Rion Gracey.
He saw the Colonel and rose to his feet with an exclamation of
surprise and pleasure. The elder man, drawing him aside, told him
the reason of his return and asked him news of June. Moving toward
the door they conversed together in lowered voices. The Colonel,
now convinced that no suspicion of the nature of Allen’s absence
had yet reached Virginia, felt his anxieties diminished. He said that
he might attend the dinner which Black Dan was giving that evening
to the Easterners. Rion was now waiting for them to come up.
Barclay and the women ought to be up at any moment; they had
been underground nearly two hours, an unusual length of time, for
even on the thousand-foot level the heat was intense.
His anxieties soothed, the Colonel left the building, his heart feeling
lighter than it had felt for two weeks.
With a step of youthful buoyancy he mounted the steep cross-streets
which connected by a series of stairs and terraces the few long
thoroughfares of the town. He was out of breath when he saw the
dark shape of the Murchison mansion standing high on its crest of
ground against a deep blue, star-dotted sky. His approach was from
the side, and that no lights appeared in any of the windows in that
part of the house did not strike him as unusual. But when he reached
the foot of the long stairway and looking up saw that there was not a
gleam of light to be seen on the entire façade, his joy suddenly died,
and in its place a dread, sharp and disturbing, seized him.
For a moment he stood motionless, staring up. The shrubs that grew
along the sloping banks of the garden rustled dryly in the autumn
night. There was something sinister in the high form of the house,
mounted aloft on its terrace, no friendly pane gleaming with
welcoming light, no sound near it but the low, occasional whispering
of dying vegetation. As he ran up the steps, his footfall sounded
singularly loud and seemed to be buffeted back from empty walls.
His first and second pull of the bell brought no response. Between
them he listened and his ear caught nothing but the stillness of
desertion. His third furious peal was answered by a distant footstep.
He heard it come shuffling along the hall, pause, and then a light
broke out through the glass fanlight above the portal. The door was
opened a crack, and through this aperture a section of the
Chinaman’s visage was revealed, lit by a warily inspecting eye.
The Colonel pushed the door violently in, sending the servant back
with it against the wall. Kicking it to behind him he demanded
between his panting breaths:
“Where’s Miss Allen?”
“She’s gone,” said the Chinaman, exceedingly startled by this violent
entry. “All gone.”
“All gone! All gone where?”
“I no savvy. The boss he gone two, thlee days. Gone San Francisco.
Miss Allen she go just now.”
“She’s only just gone? You mean she has just gone down town to
buy something or see some one?”
“No. She go ’way. She say, ‘Sing, I go ’way.’ She take a bag.”
“She’s gone with a bag. Where the devil has she gone to? Don’t be
such a damned fool! Where’d she go?”
“No savvy. She no tell me. She take bag and go just now. She give
me letter for you. I get him. He tell you.”
“You’ve got a letter for me? Why didn’t you say that before? Go get
it, and go quick.”
The Chinaman shuffled up the hall and turned into the dining-room.
The Colonel, having caught his breath, leaned against the wall under
the hall gas. He thought probably June had gone to Lake Tahoe to
visit Mitty Sullivan. Considering the situation it was the best thing she
could have done. As the servant reappeared with a letter in his hand
he said:
“When did she leave this?”
“Now,” answered the laconic Oriental. “She give him to me and say,
‘Give him Colonel Pallish. He come back Tuesday, Wednesday
mebbe. You give him letter sure; no forget.’ You come back before, I
give him now.”
The Colonel had not listened to the last phrases. He moved closer to
the gas and tore open the letter. To his surprise he saw that it was
several pages in length, covered closely with June’s fine writing. His
eye fell on the first sentence, and he uttered a sudden suppressed
sound and his body stiffened. The words were:
“Dear, darling, Uncle Jim. I who love you more than anybody in the
world am going to hurt you so much. Oh, so terribly! Will you ever
forgive me? Will you ever again think of June without sorrow and
pain?”
He stood motionless as a thing of stone, while his glance devoured
the page. He did not read every word, but from the closely written
lines sentences seemed to start out and strike his eyes. He turned
the sheet and saw farther down a paragraph that told him everything:
“The future is all dark and terrible, but I am going. I am going with
Jerry. I am going wherever he wants, I am what he wants to make
me. It’s only death that can break the spell. Good-by, dearest,
darlingest Uncle Jim. Oh, good-by! If I could only see you again for
one minute! Even when you read this and realize what I have done I
know that you will love me and make excuses for me, I who will be
no longer worthy your love or your pity.”
The Colonel’s hand with the letter crushed in it dropped to his side.
For a moment he stood rigid, his face gray in the gas light. It was too
unexpected a blow to be grasped in the first paralyzing second. Then
he turned furiously on the servant, shouting:
“Where did she go? Where did she go?”
The man cowered terrified against the wall, stammering in broken
phrases,
“I no savvy! How I savvy? She go with a bag. She say, ‘Give him the
letter’ and I give him. You read him. I no savvy any more.”
The Colonel’s hand on his chest forcing him back against the door-
post cut short his words:
“When did she go? How long ago? Answer honestly, or, by God, I’ll
kill you!”
His face added to the man’s terror, but it also steadied his shaking
nerves:
“She go not one hour; thlee-quarters. She come to me with bag and
say, ‘Good-by, Sing, I go for long time.’ She give me the letter and
say give him to you Tuesday, Wednesday. Then she go.”
“Which way?”
“I don’t see. I don’t look. I go down stairs. I go sleep on my bed. I
hear bell and wake. That’s all.”
The Colonel released him and turned to the door. The man evidently
knew no more than he said. She had been gone less than an hour.
That was all there was to tell.
As he ran down the long stairs he had no definite idea in his mind.
She had left to run away with Jerry three-quarters of an hour earlier.
That was all he thought of for the moment. Then the frosty
sharpness of the night air began to act with tonic force upon him. His
brain cleared and he remembered Rion’s words. Half an hour ago
Barclay was still in the mine. There had evidently been some delay
in his coming up. No trains left the town as late as that. June had
gone somewhere to meet him, to some place of rendezvous whence
they would probably drive into Reno. If Barclay had not yet left the
mine he could be caught, and then——
With wild speed he ran along the streets, leaping down the short
flights of steps that broke the ascending sidewalks. He thrust people
aside and rushed on, gray-faced and fiery-eyed. For the second time
in his life there was murder in his heart.
Through the darkness of his mind memories of her passed like slides
across a magic lantern. A sudden picture of her that day long ago at
the spring, when she had asked him to let her mother stay in his
cottage, rose up clear and detached on his mental vision. He heard
again the broken tones of her voice and saw her face with the tears
on it, childish and trustful, as it had been before the influence of Jerry
had blighted its youth and marred its innocence.
The fury that possessed him rose up in his throat. He could not have
spoken. He could only run on, tearing his way through the crowds on
C Street, across it to a smaller thoroughfare and down that to where
the dark mass of the Cresta Plata buildings stood out against the
night. He heard the distant hum of the machinery, and then,
unexpected and startling, the roar of men. It was like the noise when
the day shift came up and every ascending cage was packed solid
with miners.
As he approached the door the men began to come out, streams of
them, some running, others gathering in knots. Hundreds of men
poured into the night, gesticulating, shouting, congesting in black
groups, whence a broken clamor of voices rose. He realized the
strangeness of it, that something was the matter, but it was all dim
and of no importance to him. His mind held only one thought.
Rushing past them he cried:
“Barclay! Is Barclay up yet? Do you know where Barclay is?”
An Irishman, who stumbled against him in the dark, paused long
enough to shout to him:
“It’s Barclay that’s hurt. Hurry up, Colonel, they’ll be wanting you
inside. It’s a doctor I’m after. God knows if he is where they say he
is, there’s no life in him now.”
 CHAPTER VIII
THE AROUSED LION
Black Dan, as he walked to the office that Friday morning, had been
giving serious thought to the situation of his son-in-law. Mercedes
had not spent the summer in Virginia as her father had hoped and
expected. When he saw her in San Francisco, as he did every few
weeks, she talked of her delicate throat and expressed a fear of the
climate. It was evident that she could not or would not live there.
That his daughter loved her husband Black Dan had no doubt. And
as he walked to the mine that morning he was pondering a scheme
he had lately been considering of sending Jerry to San Francisco, to
be placed in charge of his large property interests. Though he
regarded his son-in-law with contemptuous dislike, he could not deny
that the young man had worked hard and faithfully all summer.
Moreover, the stealthy watch kept upon him had revealed no
irregularities in his conduct. In a place and at a time when men led
wild lives with wilder associates, Jerry’s behavior had been
exemplary. His life had been given to work and business; women
had no place in it.
With these thoughts in his mind Black Dan entered the office and
paused by his son-in-law’s desk. As he stood there a boy walked in
and handed the young man a small gray envelope that bore a
superscription in a delicate feminine hand. Black Dan also saw that
Jerry, under his unshakable sang-froid, was disconcerted. That the
receipt of the letter was disturbing to its recipient was as plain to the
older man as that the letter was from a woman.
He passed on to his own office with his mind in an entirely different
condition from what it had been when he entered the building. After
all their watchfulness, was Jerry playing at his old game? The
thought made Black Dan breathe curses into his beard. He saw
Rion, himself and the Colonel out-witted, and Jerry laughing at them
in his sleeve. And deeper than this went the enraging thought of
Mercedes supplanted by one of the women that flourish in mining
camps, birds of prey that batten on the passions of men.
He had work to do, however, and, for to-day, at least, would have to
put the matter out of his mind. Time enough when the Easterners
were gone. Black Dan, like many men of his day and kind, was
particularly anxious to impress the Easterners, and to make their
three days’ stay in the town a revel of barbaric luxury. The dinner he
was to give them that evening was to be a feast of unrivaled
splendor, every course ordered from San Francisco, the wine as
choice as any to be bought in the country, the china, glass, and silver
imported at extravagant cost from the greatest factories of Europe,
the cigars of a costly rarity, a brand especially sent from Havana for
the bonanza king and his associates.
Now from among the specimens of ore that stood along the top of
his desk he selected one of unusual form and value to give to the
most distinguished of the strangers. It was a small square of blackish
mineral on which a fine, wire-like formation of native silver had coiled
itself into a shape that resembled a rose. It had the appearance of a
cunning piece of the silversmith’s art, a flower of silver wire delicately
poised on a tiny fragment of quartz rock. Thrusting it into his coat
pocket, he left the office, on his way out passing Jerry, who was
bending studiously over his desk.
He walked rapidly up through the town, to the same livery stable to
which his son-in-law had already paid a visit. One of the diversions
to which the visitors were treated was the drive along the mountain
road to Washoe Lake. This, Black Dan had arranged, would be the
entertainment for the following morning. He with his own Kentucky
thoroughbreds, would drive the men, while the women of the party
would follow in a hired trap, drawn by the horses Jerry had ordered,
and driven by the expert whip of the stable, known as Spanish
George. Such a division of the party suited Black Dan admirably, for
he disliked women, shunning their society, and when forced into it,
becoming more somber and taciturn than ever.
His plan, however, received an unexpected check. He was told that
the horses were engaged by Mr. Barclay for that evening. Frowning
and annoyed, he demanded why that should prevent him from
having them the next morning, and received the information that Mr.
Barclay was to drive into Reno that night with them, sending them
back in the morning, when they would be too tired by the twenty-one
miles over the grade, to go out again immediately.
Black Dan stood in the doorway of the stable looking with attentive
eyes at his informant. As the man amplified his explanation with
excuses, the bonanza king said nothing. For the moment his own
thoughts were too engrossing to permit of words. A puppy that was
playing near by in the sunlight trotted toward him and bit playfully at
his toe. He turned it over with his foot, following its charmingly
awkward gambols with a pondering gaze.
“Then I suppose I can’t have Spanish George either?” he said. “Mr.
Barclay’ll take him in to drive the horses back, and he’ll take his time
about it.”
“Oh, you can have Spanish George all right,” said the stable-man,
relieved that he could give his powerful patron something he wanted.
“Mr. Barclay’s driving some one in with him. He’ll have one of the
Reno men bring the horses back.”
Black Dan looked up, his broad, dark eyes charged with almost
fierce attention.
“Who’s he driving in?” he asked.
“Don’t know, sir. He didn’t say. All he said was that he couldn’t take a
driver, as he had some one with him and he’d send the team back in
the morning with a man from Reno.”
The other looked down at the puppy, rolling it gently back and forth
with his large foot.
“When did you say he was going?” he asked.
“Six-thirty. His valises have come up already. They’re in the office
now.”
He pointed backward with his thumb toward the small, partitioned-off
box called the office. But Black Dan did not seem particularly
interested in the valises.
“Well,” he said, taking his foot off the puppy and pushing it carefully
aside, “send along the best you have with Spanish George to drive.
Be at the International at eleven sharp. I don’t want to start later than
that.”
He left the stable and walked slowly down the street toward the
Cresta Plata. His eyes were downcast, his face set in lines of
absorbed thought. Whom was Jerry driving into Reno that night?
As he walked he pieced together what he had just heard with what
he knew already. One hour before the dinner to the Easterners—at
which he was expected—Jerry had arranged to leave the town,
driving into Reno with some companion. The companion and the
gray note instantly connected themselves in Black Dan’s mind. He
felt as certain as a man could be without absolute confirmation that
Jerry was driving in with a woman. The daring insolence of it made
the blood, which moved slowly in the morose and powerful man, rise
to his head. Could it be possible that Jerry, on the way to see his
wife, was going to stop over in Reno with some woman of the
Virginia streets?
Black Dan’s swarthy skin was slightly flushed when he reached the
office. He said nothing to Jerry as he passed his desk. In his own
private office he sat still, staring in front of him at the geological map
hanging on the wall. He was slow to wrath, but his wrath, like his
love, once roused was of a primitive intensity. As he sat staring at
the map his anger gathered and grew.
At four o’clock the eastern party and their guides were due to meet in
the hoisting works for their excursion down the mine. It was nearly a
half-hour later, however, when the two ladies, who made up the
feminine portion of the party, slunk out of the spacious dressing-
rooms, giggling and blushing in their male attire. Jerry, Marsden the
foreman, and one of the shift bosses, were lounging about the mouth
of the shaft waiting for them. There were greetings and laughter, the
women hugging themselves close in the long overcoats they wore
against the chill of the downward passage, and pulling over their hair
the shapeless cloth caps they had been given for head-gear.
Through the wide opening that led to the dumps the figure of Black
Dan, dark against the brilliance of the afternoon, could be seen
walking on the car tracks with the rest of the party. In the muddy
overalls, long boots and soft felt hat which was the regulation
underground dress of the men, he presented the appearance of
some black-browed, heavily bearded pirate in the garb of a tramp.
As the cage slid up to the shaft mouth, he entered the building, gave
the embarrassed women an encouraging nod, and selected a lantern
from a collection of them standing in a corner.
With little cries of apprehension the women stepped on the flat
square of flooring, their three escorts ranged closely round them, the
signal to descend, was given, and the cage dropped quickly out of
sight into the steaming depths. Black Dan, Barney Sullivan and the
strangers were to descend on the cage in the next compartment, and
while they waited for it to come up, stood talking of the formations of
the mineral, how it had been found and of the varying richness of the
ore-bodies. Suddenly Black Dan thought of his specimen, which had
come from a part of the mine they were to visit first, and turning went
into the men’s dressing-room, where he had left it in his coat pocket.
His clothes had been hung on the last of a line of pegs along the
wall. To this he went, and, ignorant of the fact that Jerry had
undressed after him, thrust his hand into the pocket of what he
thought was his own coat. Instead of the stone his fingers
encountered a letter. He drew it out and saw that it was the one he
had seen handed to his son-in-law a few hours before.
At once he drew the paper from the envelope. No qualm of
conscience deterred him; instead he experienced a sense of
satisfaction that his uncertainty should be thus simply brought to an
end. His eye traveled over the few lines, instantly grasping their
meaning. He knew the signature. Jerry was not intriguing with a
common woman of the town; he was deserting his wife with a girl,
hitherto of unspotted reputation, and for years beloved by Rion. It
meant ruin and misery for the two human beings nearest to the
bonanza king’s heart.
For a moment he stood motionless, the letter in his hand, and before
his eyes he saw red. Then it cleared away. He put the paper back in
its envelope and thrust it in his pocket. When he came out into the
shaft house Barney Sullivan noticed that his face was reddened and
that the whites of his eyes were slightly bloodshot. One of the
strangers rallied him on his absence, which had been of some
minutes’ duration, and he made no answer, simply motioning them to
get on the cage with an imperious movement of his head.
The shaft of the Cresta Plata was over two thousand feet in depth,
and the heat of the lower levels was terrific. Here the miners, naked,
save for a cap, breechclout, and canvas shoes, worked twenty-
minute shifts, unable to stand the fiery atmosphere for longer. Cold
air was pumped down to them from the surface, the pipes that
carried it following the roofs of the long, dark tunnels, their mouths
blowing life-giving coolness into stopes where the men could not
touch their metal candlesticks, and the iron of the picks grew hot.
There were places where the drops that fell from the roof raised
blisters on the backs they touched. On most of these lower levels
there was much water, its temperature sometimes boiling. The
miners of the Cresta Plata had a saying that no man had ever fallen
into water that reached to his hips and lived. At the bottom of the
shaft—the “sump” in mining parlance—was a well of varying depths
which perpetually exhaled a scalding steam.
Black Dan took his guests to the fifteen-hundred-foot level, whence
the greatest riches of the mine had been taken. He was more than
usually silent as they walked from tunnel to tunnel and drift to drift.
Barney Sullivan was the cicerone of the party, explaining the
formation, talking learnedly of the dip of the vein, holding up his
lantern to let its gleam fall on the dark bluish “breast” into which the
miners drove their picks with a gasp of expelled breath. Nearly an
hour had passed when Black Dan, suddenly drawing him back,
whispered to him that he was going up to the eight-hundred-foot
level to see Jerry, to whom he wished to give some instructions
about the dinner that evening. Barney, nodding his comprehension,
moved on with the guests, and Black Dan walked back to the station.
As he went up in the cage he passed level after level, like the floors
of a great underground building. Yellow lights gleamed through the
darkness on the circular forms of west timbers, hollowed caves
trickling with moisture, car tracks running into blackness. Each floor
was peopled with wild, naked shapes, delving ferociously in this
torrid inferno. At the eight-hundred-foot level he got off, the bell rang,
and the empty cage went sliding up. The landing on to which he
stepped was deserted, and he walked up one of the tunnels that
branched from it, called to a pick-boy, whom he saw in the distance,
that he wanted Mr. Barclay found and sent to him at once. The figure
of the boy scudded away into the darkness, and Black Dan went
back to the landing.
It was an open space, a small, subterranean room, the lanterns
fastened on its walls gilding with their luster the pools of water on the
muddy floor. There were boxes used for seats standing about, and
on pegs in the timbers the miners’ coats hung. Where the shaft
passed down there were several square openings—larger than
ordinary doorways, iron-framed and with plates of iron set into the
moist ground—which gave egress to the cages. Now there was only
a black void there, the long shaft stretching hundreds of feet upward
and downward.
Black Dan sat on a box, waiting. Afar off from some unseen tunnel
he could hear the faint sound of voices. Near by, sharply clear in the
stifling quiet, came the drip of water from the roof. It was still very
hot, a moist, suffocating heat, regarded by the miners as cool after
the fiery depths below. He pushed back his hat and wiped the sweat
from his face. His eyes, as he waited, kept watch on the openings of
the three tunnels that diverged from this central point.
One of them was an inky arch in a frame of timbers. In the distance
of the others lights gleamed. Now and then a bare body, streaming
with perspiration, came into view pushing an ore car. With an
increasing rattle it was rolled to the shaft opening and on to a waiting
cage which slid up. The miner slouched back into the gloom, the
noise of the empty car he propelled before him gradually dying away.
Black Dan could hear again the voices and then, muffled by earth
and timbers, the thud of the picks. Sitting on an upturned box—the
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com