Get the full ebook with Bonus Features for a Better Reading Experience on ebookgate.

com

Linear programming with MATLAB 1st Edition Michael

C. Ferris

https://ebookgate.com/product/linear-programming-with-
matlab-1st-edition-michael-c-ferris/

OR CLICK HERE

DOWLOAD NOW

Download more ebook instantly today at https://ebookgate.com

Instant digital products (PDF, ePub, MOBI) available
Download now and explore formats that suit you...

Linear Control System Analysis and Design with MATLAB

Sixth Edition Constantine H. Houpis

https://ebookgate.com/product/linear-control-system-analysis-and-
design-with-matlab-sixth-edition-constantine-h-houpis/

ebookgate.com

Linear Control System Analysis and Design with MATLAB 6th

Edition Constantine H. Houpis

https://ebookgate.com/product/linear-control-system-analysis-and-
design-with-matlab-6th-edition-constantine-h-houpis/

ebookgate.com

Programming embedded systems with C and GNU development

tools 2nd Edition Michael Barr

https://ebookgate.com/product/programming-embedded-systems-with-c-and-
gnu-development-tools-2nd-edition-michael-barr/

ebookgate.com

Programming With C 3rd Edition Bl Juneja

https://ebookgate.com/product/programming-with-c-3rd-edition-bl-
juneja/

ebookgate.com
CryENGINE Game Programming with C C and Lua 1st Edition
Lundgren

https://ebookgate.com/product/cryengine-game-programming-with-c-c-and-
lua-1st-edition-lundgren/

ebookgate.com

MATLAB Graphical Programming Practical hands on MATLAB

solutions 1st Edition Cesar Perez Lopez

https://ebookgate.com/product/matlab-graphical-programming-practical-
hands-on-matlab-solutions-1st-edition-cesar-perez-lopez/

ebookgate.com

Programming with ANSI C 2nd ed Edition Trivedi

https://ebookgate.com/product/programming-with-ansi-c-2nd-ed-edition-
trivedi/

ebookgate.com

Fundamentals of Computer Programming with C The Bulgarian

C Book 1st Edition Svetlin Nakov

https://ebookgate.com/product/fundamentals-of-computer-programming-
with-c-the-bulgarian-c-book-1st-edition-svetlin-nakov/

ebookgate.com

Programming Windows Store Apps with C 1st Edition Matthew

Baxter-Reynolds

https://ebookgate.com/product/programming-windows-store-apps-
with-c-1st-edition-matthew-baxter-reynolds/

ebookgate.com
MP07_Ferris_FMA.qxp 10/4/2007 2:22 PM Page 1

LINEAR PROGRAMMING
WITH MATLAB
MP07_Ferris_FMA.qxp 10/4/2007 2:22 PM Page 2

MPS-SIAM Series on Optimization

This series is published jointly by the Mathematical Programming Society and the Society
for Industrial and Applied Mathematics. It includes research monographs, books on
applications, textbooks at all levels, and tutorials. Besides being of high scientific quality,
books in the series must advance the understanding and practice of optimization. They
must also be written clearly and at an appropriate level.

Editor-in-Chief
Philippe Toint, University of Namur (FUNDP)

Editorial Board
Oktay Gunluk, IBM T.J. Watson Research Center
Matthias Heinkenschloss, Rice University
C.T. Kelley, North Carolina State University
Adrian S. Lewis, Cornell University
Pablo Parrilo, Massachusetts Institute of Technology
Daniel Ralph, University of Cambridge
Mike Todd, Cornell University
Laurence Wolsey, Université Catholique de Louvain
Yinyu Ye, Stanford University

Series Volumes
Ferris, Michael C., Mangasarian, Olvi L., and Wright, Stephen J., Linear Programming with
MATLAB
Attouch, Hedy, Buttazzo, Giuseppe, and Michaille, Gérard, Variational Analysis in Sobolev
and BV Spaces: Applications to PDEs and Optimization
Wallace, Stein W. and Ziemba, William T., editors, Applications of Stochastic Programming
Grötschel, Martin, editor, The Sharpest Cut: The Impact of Manfred Padberg and His Work
Renegar, James, A Mathematical View of Interior-Point Methods in Convex Optimization
Ben-Tal, Aharon and Nemirovski, Arkadi, Lectures on Modern Convex Optimization:
Analysis, Algorithms, and Engineering Applications
Conn, Andrew R., Gould, Nicholas I. M., and Toint, Phillippe L., Trust-Region Methods
MP07_Ferris_FMA.qxp 10/4/2007 2:22 PM Page 3

LINEAR PROGRAMMING
WITH MATLAB

Michael C. Ferris
Olvi L. Mangasarian
Stephen J. Wright
University of Wisconsin–Madison
Madison, Wisconsin

Society for Industrial and Applied Mathematics Mathematical Programming Society

Philadelphia Philadelphia
MP07_Ferris_FMA.qxp 10/4/2007 2:22 PM Page 4

Copyright © 2007 by the Society for Industrial and Applied Mathematics and the
Mathematical Programming Society

10 9 8 7 6 5 4 3 2 1

All rights reserved. Printed in the United States of America. No part of this book may be
reproduced, stored, or transmitted in any manner without the written permission of the
publisher. For information, write to the Society for Industrial and Applied Mathematics,
3600 Market Street, 6th floor, Philadelphia, PA 19104-2688 USA.

Trademarked names may be used in this book without the inclusion of a trademark symbol.
These names are used in an editorial context only; no infringement of trademark is intended.

Maple is a registered trademark of Waterloo Maple, Inc.

MATLAB is a registered trademark of The MathWorks, Inc. For MATLAB product information,
please contact The MathWorks, Inc., 3 Apple Hill Drive, Natick, MA 01760-2098 USA,
508-647-7000, Fax: 508-647-7101, info@mathworks.com, www.mathworks.com.

Library of Congress Cataloging-in-Publication Data

Ferris, Michael C.
Linear programming with MATLAB / Michael C. Ferris, Olvi L. Mangasarian, Stephen J. Wright.
p. cm. — (MPS-SIAM series on optimization ; 7)
Includes bibliographical references and index.
ISBN 978-0-898716-43-6 (alk. paper)
1. Linear programming—Data processing. 2. MATLAB. 3. Mathematical optimization. 4. Algebras,
Linear. I. Mangasarian, Olvi L., 1934- II. Wright, Stephen J., 1960- III. Title.

QA402.5.F425 2007
519.7’2—dc22
2007061748

is a registered trademark. is a registered trademark.

MP07_Ferris_FMA.qxp 10/4/2007 2:22 PM Page 5

To Jane, Claire, and Jill


MP07_Ferris_FMA.qxp 10/4/2007 2:22 PM Page 6
 Contents

Preface xi

1 Introduction 1
1.1 An Example: The Professor’s Dairy . . . . . . . . . . . . . . . . . . . 2
1.1.1 The Setup . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Formulating the Problem and a Graphical Solution . . . . 2
1.1.3 Changing the Problem . . . . . . . . . . . . . . . . . . . 4
1.1.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.1 The Diet Problem . . . . . . . . . . . . . . . . . . . . . 8
1.3.2 Linear Surface Fitting . . . . . . . . . . . . . . . . . . . 9
1.3.3 Load Balancing Problem . . . . . . . . . . . . . . . . . . 10
1.3.4 Resource Allocation . . . . . . . . . . . . . . . . . . . . 10
1.3.5 Classification . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.6 Minimum-Cost Network Flow . . . . . . . . . . . . . . . 12
1.4 Algorithms and Complexity . . . . . . . . . . . . . . . . . . . . . . . 14
1.4.1 The Simplex Method . . . . . . . . . . . . . . . . . . . . 14
1.4.2 Interior-Point Methods . . . . . . . . . . . . . . . . . . . 15

2 Linear Algebra: A Constructive Approach 17

2.1 Jordan Exchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Linear Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3 Matrix Inversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Exact Solution of m Equations in n Unknowns . . . . . . . . . . . . . 32
2.5 Solving Linear Equations Efficiently . . . . . . . . . . . . . . . . . . 39
2.6 LU Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3 The Simplex Method 45

3.1 A Simple Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2 Vertices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.3 The Phase II Procedure . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.4 The Phase I Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.5 Finite Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

vii
viii Contents

3.5.1 The Nondegenerate Case . . . . . . . . . . . . . . . . . 65

3.5.2 Cycling . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.5.3 The General Case . . . . . . . . . . . . . . . . . . . . . 67
3.6 Linear Programs in Nonstandard Form . . . . . . . . . . . . . . . . . 72
3.6.1 Transforming Constraints and Variables . . . . . . . . . . 72
3.6.2 Scheme I . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.6.3 Scheme II . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . 86

4 Duality 89
4.1 Duality and Rank in Linear Systems . . . . . . . . . . . . . . . . . . 89
4.2 Duality in Linear Programming . . . . . . . . . . . . . . . . . . . . . 94
4.3 Interpretation of Linear Programming Duality . . . . . . . . . . . . . 96
4.4 Duality Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.5 KKT Optimality Conditions . . . . . . . . . . . . . . . . . . . . . . . 100
4.6 Dual Simplex Method . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.7 General Linear Programs . . . . . . . . . . . . . . . . . . . . . . . . 107
4.8 Big M Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.9 Applications of Duality . . . . . . . . . . . . . . . . . . . . . . . . . 112

5 Solving Large Linear Programs 117

5.1 Foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.1.1 Basic Feasible Solutions and Basis Matrices . . . . . . . 118
5.1.2 Geometric Viewpoint . . . . . . . . . . . . . . . . . . . 121
5.2 The Revised Simplex Method . . . . . . . . . . . . . . . . . . . . . . 123
5.2.1 Upper and Lower Bounds . . . . . . . . . . . . . . . . . 129
5.2.2 Generating Basic Feasible Solutions . . . . . . . . . . . 134
5.2.3 Basis Updates . . . . . . . . . . . . . . . . . . . . . . . 139
5.2.4 Advanced Pivot Selection Mechanisms . . . . . . . . . . 142
5.3 Network Flow Problems . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.3.1 Minimum-Cost Network Flow . . . . . . . . . . . . . . . 144
5.3.2 Shortest-Path Problem . . . . . . . . . . . . . . . . . . . 145
5.3.3 Max-Flow Problem . . . . . . . . . . . . . . . . . . . . 146
5.3.4 Transportation Problem . . . . . . . . . . . . . . . . . . 147
5.3.5 Assignment Problem . . . . . . . . . . . . . . . . . . . . 149
5.3.6 Network Simplex Method . . . . . . . . . . . . . . . . . 149

6 Sensitivity and Parametric Linear Programming 151

6.1 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
6.2 Adding New Variables or Constraints . . . . . . . . . . . . . . . . . . 155
6.3 Parametric Optimization of the Objective Function . . . . . . . . . . . 158
6.4 Parametric Optimization of the Right-Hand Side . . . . . . . . . . . . 164

7 Quadratic Programming and Complementarity Problems 169

7.1 Nonlinear Programs: Optimality Conditions . . . . . . . . . . . . . . 169
7.2 Quadratic Programming . . . . . . . . . . . . . . . . . . . . . . . . . 172
Contents ix

7.2.1 Basic Existence Result . . . . . . . . . . . . . . . . . . . 172

7.2.2 KKT Conditions . . . . . . . . . . . . . . . . . . . . . . 173
7.2.3 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
7.3 Linear Complementarity Problems . . . . . . . . . . . . . . . . . . . 177
7.4 Lemke’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
7.5 Bimatrix Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
7.5.1 Computing Nash Equilibria . . . . . . . . . . . . . . . . 186
7.5.2 Zero-Sum Games As Dual Linear Programs . . . . . . . 192

8 Interior-Point Methods 195

8.1 Motivation and Outline . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.2 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
8.3 Primal-Dual Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 201
8.3.1 An Affine-Scaling Approach . . . . . . . . . . . . . . . . 202
8.3.2 Path-Following Methods . . . . . . . . . . . . . . . . . . 204
8.3.3 Solution of the Linear System at Each Interior-Point
Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . 208
8.3.4 Practical Primal-Dual Methods . . . . . . . . . . . . . . 209
8.4 Interior-Point vs. Simplex . . . . . . . . . . . . . . . . . . . . . . . . 212
8.5 Extension to Quadratic Programming . . . . . . . . . . . . . . . . . . 212

9 Approximation and Classification 217

9.1 Minimax Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
9.2 Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
9.2.1 Chebyshev Approximation . . . . . . . . . . . . . . . . . 219
9.2.2 L1 Approximation . . . . . . . . . . . . . . . . . . . . . 221
9.2.3 Approximate Solutions to Systems with Inequality
Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 223
9.2.4 Least-Squares Problems . . . . . . . . . . . . . . . . . . 224
9.3 Huber Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
9.4 Classification Problems . . . . . . . . . . . . . . . . . . . . . . . . . 230

A Linear Algebra, Convexity, and Nonlinear Functions 237

A.1 Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
A.2 Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
A.3 Smooth Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
A.4 Convex Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
A.5 Quadratic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
A.6 Norms and Order Notation . . . . . . . . . . . . . . . . . . . . . . . 247
A.7 Taylor’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

B Summary of Available MATLAB Commands 251

B.1 Basic MATLAB Operations . . . . . . . . . . . . . . . . . . . . . . . 251
B.2 MATLAB Functions Defined in This Book . . . . . . . . . . . . . . . 252
x Contents

Bibliography 257

Index 261
 Preface

This text has grown over many years from a set of class notes for an undergraduate linear
programming course offered at the University of Wisconsin-Madison. Though targeted to
Computer Science undergraduates, the course has attracted both undergraduates and be-
ginning graduate students from many other departments, including Industrial and Systems
Engineering, Statistics, and Mathematics. The course aims to provide a one-semester ele-
mentary introduction to linear programming formulations, algorithms, computations, and
applications. Only basic knowledge of linear algebra and calculus is required.
One feature of our approach is the use of MATLAB codes to demonstrate the com-
putational aspects of the course, from the elementary manipulations that form the building
blocks of algorithms to full implementations of revised simplex and interior-point methods.
(The latter are clearly not robust or efficient enough to solve larger practical problems,
but they do illustrate the basic principles of the computational methods in question.) The
MATLAB codes (and associated mex files) are distributed on the web site associated with
the book: www.siam.org/books/mp07.
We have included a chapter on quadratic programs and complementarity problems,
which are topics whose importance in a number of application areas appears to be growing
by the day. The final chapter deals with approximation and classification problems, which
are of interest to statisticians and others, showing how these problems can be formulated and
solved as linear or quadratic programs. An earlier chapter deals with the topic of duality,
which is of interest not only because of the insight it provides into the beautiful theory
underlying linear programming but also because of its usefulness in formulating practical
problems. (The dual of a problem may be easier to solve than the primal, or it might provide
a bound on the optimal solution value or other useful information.)
A one-semester undergraduate class should include most of the chapters in the text.
If time is pressing, some of the later chapters could be omitted in part or in their entirety.
However, we believe that all topics covered are interesting and relevant to the intended
student audience, and so we hope that most teachers can find a way to incorporate them into
their curriculum.
We thank the students and colleagues who have given us feedback on the manuscript
during its development, particularly our colleague Bob Meyer. We are also grateful to the
referees of the manuscript who read it thoroughly and provided valuable suggestions for
improvement, most of which we adopted. Finally, we thank our wives and families for their
love and support over many years.
Madison, Wisconsin, USA
Spring 2007
xi
Chapter 1

Introduction

Nothing happens in the universe that does not have a sense of either certain maximum or
minimum. L. Euler, Swiss Mathematician and Physicist, 1707–1783

Optimization is a fundamental tool for understanding nature, science, engineering,

economics, and mathematics. Physical and chemical systems tend to a state that minimizes
some measure of their energy. People try to operate man-made systems (for example, a
chemical plant, a cancer treatment device, an investment portfolio, or a nation’s economy)
to optimize their performance in some sense. Consider the following examples:

1. Given a range of foods to choose from, what is the diet of lowest cost that meets an
individual’s nutritional requirements?

2. What is the most profitable schedule an airline can devise given a particular fleet of
planes, a certain level of staffing, and expected demands on the various routes?

3. Where should a company locate its factories and warehouses so that the costs of
transporting raw materials and finished products are minimized?

4. How should the equipment in an oil refinery be operated, so as to maximize rate of

production while meeting given standards of quality?

5. What is the best treatment plan for a cancer patient, given the characteristics of the
tumor and its proximity to vital organs?

Simple problems of this type can sometimes be solved by common sense, or by using
tools from calculus. Others can be formulated as optimization problems, in which the goal
is to select values that maximize or minimize a given objective function, subject to certain
constraints. In the next section, we show how a practical problem can be formulated as a
particular type of optimization problem known as a linear program.

1
2 Chapter 1. Introduction

1.1 An Example: The Professor’s Dairy

1.1.1 The Setup
University professors sometimes engage in businesses to make a little extra cash. Professor
Snape and his family run a business that produces and sells dairy products from the milk
of the family cows, Daisy, Ermentrude, and Florence. Together, the three cows produce 22
gallons of milk each week, and Snape and his family turn the milk into ice cream and butter
that they then sell at the Farmer’s Market each Saturday morning.
The butter-making process requires 2 gallons of milk to produce one kilogram of
butter, and 3 gallons of milk is required to make one gallon of ice cream. Professor Snape
owns a huge refrigerator that can store practically unlimited amounts of butter, but his freezer
can hold at most 6 gallons of ice cream.
Snape’s family has at most 6 hours per week in total to spend on manufacturing their
delicious products. One hour of work is needed to produce either 4 gallons of ice cream or
one kilogram of butter. Any fraction of one hour is needed to produce the corresponding
fraction of product.
Professor Snape’s products have a great reputation, and he always sells everything
he produces. He sets the prices to ensure a profit of $5 per gallon of ice cream and $4 per
kilogram of butter. He would like to figure out how much ice cream and butter he should
produce to maximize his profit.

1.1.2 Formulating the Problem and a Graphical Solution

The first step in formulating this problem is to identify the two variables, which are the
quantities that we are able to vary. These are the number of gallons of ice cream, which we
denote by x, and the number of kilograms of butter, which we denote by y. Next, we figure
out how the objective function depends on these variables. We denote the objective (which
in this case is the profit) by z, and note that it is simply z = 5x + 4y dollars in this example.
Since we aim to maximize the production, it is generally in our interest to choose x
and y as large as possible. However, the constraints on production mentioned above prevent
us from making these variables too large. We now formulate the various constraints in the
description above algebraically.

• The 6-gallon constraint on freezer capacity causes us to impose the constraint x ≤ 6.

• The total amount of labor required to produce x gallons of ice cream and y kilograms
of butter is .25x + y. Since the family can labor for a total of at most 6 hours during
the week, we have the constraint .25x + y ≤ 6.

• We look at the amount of milk needed by the production process. The total number
of gallons of milk used is 3x + 2y, and since there are 22 gallons of milk available,
we have the constraint 3x + 2y ≤ 22.

• Finally, the problem must include the simple constraints x ≥ 0, y ≥ 0, because it

does not make sense to produce negative amounts of ice cream or butter.
1.1. An Example: The Professor’s Dairy 3

labor constraint (.25 x + y <= 6) freezer constraint (x <= 6)

y (butter) solution (4,5)

milk constraint (3x+2y <= 22)

z=
40
x (ice cream)
z=

z=
z=
10

30
20

Figure 1.1. The Professor’s dairy: Constraints and objective.

Summarizing, we can express the linear program mathematically as follows:

max z = 5x + 4y
x,y
subject to x ≤ 6,
.25x + y ≤ 6, (1.1)
3x + 2y ≤ 22,
x, y ≥ 0.
Figure 1.1 illustrates this problem graphically, plotting the variable x along the hori-
zontal axis and y along the vertical axis. Each constraint is represented by a line, the shaded
side of the line representing the region of the (x, y) plane that fails to satisfy the constraint.
For example, the constraint 3x + 2y ≤ 22 is represented by the line 3x + 2y = 22 (obtained
by replacing the inequality by an equality), with the “upper” side of the line shaded. In
general, we can determine which side of the line satisfies the constraint and which does not
by picking a point that does not lie on the line and determining whether or not the constraint
is satisfied at this point. If so, then all points on this side of the line are feasible; if not, then
all points on this side of the line are infeasible.
The set of points satisfying all five of the constraints is known as the feasible region.
In this problem the feasible region is the five-sided polygonal region in the middle of the
figure.
The linear programming problem is to the find a point in this feasible region that
maximizes the objective z = 5x + 4y. As a step towards this goal, we plot in Figure 1.1
a dotted line representing the set of points at which z = 20. This line indicates feasible
points such as (x, y) = (0, 5) and (x, y) = (2, 2.5) that yield a profit of $20. Similarly,
we plot the line z = 5x + 4y = 30—the set of points that achieves a profit of $30. Note
4 Chapter 1. Introduction

that this line (and all other lines of constant z) is parallel to the line z = 20. In fact, we can
maximize profit over the feasible region by moving this line as far as possible to the right
while keeping some overlap with the feasible region and keeping it parallel to the z = 20
line. It is not difficult to see that this process will lead us to a profit of z = 40 and that
this line intersects the feasible region at the single point (x, y) = (4, 5). Note that this
point is a “corner point” of the feasible region, corresponding to the point at which two
of the constraints—the limit of milk supply and the limit on labor supply—are satisfied as
equalities.

1.1.3 Changing the Problem

The graphical representation of Figure 1.1 can be used to see how the solution changes when
the data is changed in certain ways. An investigation of this type is known as sensitivity
analysis and will be discussed in Chapter 6. We discuss two possible changes to the example
problem here. A first time reader may skip this section without loss of continuity since it is
meant primarily as an intuitive graphical introduction to duality and sensitivity.
First, we look at what happens if Professor Snape decides to increase the price of
ice cream, while leaving the price of butter (and all the other problem data) the same. We
ask the question, How much can we increase the price of ice cream without changing the
solution (4, 5)? It is intuitively clear that if the profit on ice cream is much greater than on
butter, it would make sense to make as much ice cream as possible subject to meeting the
constraints, that is, 6 gallons. Hence, if the price of ice cream increases by more than a
certain amount, the solution will move away from the point (4, 5).
Suppose for instance that we increase the profit on ice cream to $5.50, so that the
objective function becomes z = 5.5x + 4y. If we plot the contours of this new objective
(see Figure 1.2), we find that they are rotated slightly in the clockwise direction from the
contours in Figure 1.1. It is clear that for a $42 profit, (4, 5) is still the optimum. However,
if the profit on ice cream is increased further, the contours will eventually have exactly the
same slope as the “milk” constraint, at which point every point on the line joining (4, 5) to
(6, 2) will be a solution. What ice cream profit p will make the contours of the objective
z = px + 4y parallel to the line 3x + 2y = 22? By matching the slopes of these two lines,
we find that the operative value is p = 6. If the price of ice cream is slightly higher than 6,
the point (6, 2) will be the unique optimum.
Exercise 1-1-1. Plot a figure like Figures 1.1 and 1.2 for the case in which the objective is
z = 8x + 4y, while the constraints remain the same. Verify from your figure that (6, 2) is
the optimum.
Returning to the original problem, we could ask a slightly different question. Suppose
that Professor Snape’s neighbor, Professor Crouch, has some excess milk and is offering to
sell it to Snape for $1 per gallon. Given that Snape still wants to maximize his profits, and
given that his other constraints are still in place (labor and freezer capacity), should he buy
any milk from Crouch and, if so, how much?
To answer this question, we note first that if Snape purchases c gallons, the milk
constraint becomes 3x + 2y ≤ 22 + c. Graphically, the boundary of this constraint shifts
upward and to the right, as we see in Figure 1.3. Provided c is not too large, the contours of
the objective will not be greatly affected by this change to the problem, and so the solution
1.1. An Example: The Professor’s Dairy 5

labor constraint (.25 x + y <= 6) freezer constraint (x <= 6)

y (butter) solution (4,5)

milk constraint (3x+2y <= 22)

z=
40
x (ice cream)
z=

z=
20

30
Figure 1.2. The Professor’s dairy: After increasing the profit on ice cream to
$5.50, the objective contours rotate slightly clockwise, but the optimum is still (4, 5).

labor constraint (.25 x + y <= 6) freezer constraint (x <= 6)

solution (4+.4c,5−.1c)
y (butter)

milk constraint (3x+2y <= 22+c)

x (ice cream)

Figure 1.3. The Professor’s dairy: If the professor purchases c gallons from his
neighbor, the milk constraint shifts upward and to the right.
6 Chapter 1. Introduction

will still occur at the intersection of the labor constraint with the milk constraint, that is, at
the point (x, y) that satisfies the following two equalities:

.25x + y = 6,
3x + 2y = 22 + c.

The solution is
(x, y) = (4 + .4c, 5 − .1c),
and the objective function value at this point (allowing for the $1 per gallon purchase price
of milk from Crouch) is

z = 5x + 4y − c = 5(4 + .4c) + 4(5 − .1c) − c = 40 + .6c.

It follows that it is definitely to Snape’s advantage to buy some milk from Crouch, as he
earns an extra 60 cents in profit for each gallon purchased.
However, if c is too large, the solution will no longer be at the intersection of the
labor and milk constraints, and there is no further advantage to be gained. This happens
when the milk constraint is shifted so far that it intersects with both the labor limit and the
freezer limit at the point (6, 4.5), which is true when c = 5. As c increases above this value,
the solution stays at (6, 4.5) while the profit actually starts to decline, as Snape is buying
surplus milk unnecessarily without producing any more of either butter or ice cream.
Analysis of this type will be discussed further when we cover the subject of duality
in Chapter 4.
The graphical analysis used in this section is sufficient for understanding problems
with two variables. However, when extra variables are added (for example, if the professor
decides to make cottage cheese and gourmet yogurt as well), it is hard to solve or analyze the
problem using graphical techniques alone. This book describes computational techniques,
motivated by the graphical analysis above, that can be used to solve problems with many
variables and constraints. Solution of this problem using an algebraic approach, namely the
simplex method, is given in Section 3.

1.1.4 Discussion
The example of this section has three important properties.

• Its variables (the amounts of ice cream and butter to produce) are continuous variables.
They can take on any real value, subject to satisfying the bounds and constraints.

• All constraints and bounds involve linear functions of the variables. That is, each
term of the sum is either a constant or else a constant multiple of one of the variables.

• The objective function—profit, in this case—is also a linear function of the variables.

Problems with these three essential properties are known as linear programming prob-
lems or linear programs. Most of our book is devoted to algorithms for solving this class
of problems. Linear programming can be extended in various ways to give broader classes
of optimization problems. For instance, if we allow the objective function to be a quadratic
1.2. Formulations 7

function of the variables (but still require the constraint to be linear and the variables to
be continuous), we obtain quadratic programming problems, which we study in Chapter 7.
If we allow both constraints and objective to be nonlinear functions (but still require con-
tinuous variables), the problem becomes a nonlinear program. If we restrict some of the
variables to take on integer values, the problem becomes an integer program. We give
several references for nonlinear and integer programming in the Notes and References at
the end of this chapter.
Since 1947, when George B. Dantzig proposed his now classic simplex method for
solving linear programs, the utilization of linear programming as a tool for modeling and
computation has grown tremendously. Besides becoming a powerful tool in the area for
which it was originally designed (economic planning), it has found a myriad of applications
in such diverse areas as numerical analysis, approximation theory, pattern recognition, and
machine learning. It has become a key tool in the important disciplines of operations
research and management science.

1.2 Formulations
Throughout this book, we will refer to the following form of the linear program as the
standard form:
min z = p1 x1 + · · · + pn xn
x1 ,x2 ,...,xn
subject to A11 x1 + ··· + A1n xn ≥ b1 ,
.. .. .. .. (1.2)
. . . .
Am1 x1 + ··· + Amn xn ≥ bm ,
x1 , x2 , . . . , xn ≥ 0.
By grouping the variables x1 , x2 , . . . , xn into a vector x and constructing the following
matrix and vectors from the problem data,
     
A11 · · · A1n b1 p1
 .. .   .   
A= . . .. ..  , b =  ..  , p =  ...  ,
Am1 · · · Amn bm pn
we can restate the standard form compactly as follows:
min z = p x
x
subject to Ax ≥ b, x ≥ 0,
where p denotes the transpose of the column vector p, which is known as the cost vector.
Every linear program can be put into this standard form. We show in Chapter 3
how problems with equality constraints, free variables, and so on can be reformulated as
standard-form problems. Problem (1.1) of the previous section can be expressed in standard
form by setting x to be the vector made up of the two scalars x and y, while
   
1 0 6  
    5
A = − .25 1 , b=− 6 , p=− .
4
3 2 22
8 Chapter 1. Introduction

To perform this conversion, we changed “≤” inequality constraints into “≥” inequalities by
simply multiplying both sides by −1. We also noted that maximization of a function (which
we do in (1.1)) is equivalent to minimization of the negation of this function, which is why
we have negative entries in p above.
In Chapter 5 we introduce another formulation in which all the general constraints are
assumed to be equality constraints. This is known as the canonical form and is written as
follows:
min z = p x
subject to Ax = b, x ≥ 0.
As with the standard form, any linear program can be put into this form by appropriate
transformations of the constraints and variables. We could express our example (1.1) in
canonical form by first replacing (x, y) by (x1 , x2 ) in (1.1) and then introducing three slack
variables x3 , x4 , and x5 to represent the amount by which the right-hand sides exceed the
left-hand sides of the three constraints. We then obtain the following formulation:

min z = −5x1 − 4x2

x
subject to x1 + x3 = 6,
.25x1 + x2 + x4 = 6,
3x1 + 2x2 + x5 = 22,
x1 , x2 , x3 , x4 , x5 ≥ 0.

We can verify that the problem is in canonical form by setting

   
    5 x1
1 0 1 0 0 6 4  x2 
   
A = .25 1 0 1 0 , b =  6 , p = − 
0 , x= 
 x3  .
3 2 0 0 1 22 0  x4 
0 x5

1.3 Applications
In this section, we discuss several other practical problems that can be formulated as linear
programs.

1.3.1 The Diet Problem

In an early application, linear programming was used to determine the daily diet for a person.
From among a large number of possible foods, a diet was determined that achieved all the
nutritional requirements of the individual while minimizing total cost.
To formulate as a linear program, we suppose that the n possible foods are indexed by
j = 1, 2, . . . , n and that the m nutritional categories are indexed by i = 1, 2, . . . , m. We
let xj be the amount of food j to be included in the diet (measured in number of servings),
and denote by pj the cost of one serving of food j . We let bi denote the minimum daily
requirement of nutrient i and Aij be the amount of nutrient i contained in one serving of
food j . By assembling this data into matrices and vectors in the usual way, we find that the
1.3. Applications 9

linear program to determine the optimal diet can be formulated as follows:

min z = p x
x
subject to Ax ≥ b, x ≥ 0.

The bounds x ≥ 0 indicate that only nonnegative amounts of each food will be considered,
while the “≥” inequality constraints require the diet to meet or exceed the nutritional re-
quirements in each category i = 1, 2, . . . , m. If we wish to place an upper limit of dj on
the number of servings of food j to be included in the diet (to ensure that the diet does
not become too heavy on any one particular food), we could add the constraints xj ≤ dj ,
j = 1, 2, . . . , n, to the model.

1.3.2 Linear Surface Fitting

Suppose that we have a set of observations (Ai· , bi ), i = 1, 2, . . . , m, where each Ai· is a
(row) vector with n real elements, and each bi is a single real number. We would like to find
a vector x ∈ Rn and a constant γ such that

Ai· x + γ ≈ bi for each i = 1, 2, . . . , m.

The elements of the vector x can be thought of as “weights” that are applied to the components
of Ai· to yield a prediction of each scalar bi . For example, m could be the number of people
in a population under study, and the components of each Ai· could represent the income
of person i, the number of years they completed in school, the value of their house, their
number of dependent children, and so on. Each bi could represent the amount of federal
income tax they pay.
To find the “best” pair (x, γ ), we need to measure the misfit between Ai· x + γ and bi
over all the i. One possible technique is to sum the absolute values of all the mismatches,
that is,
m
|Ai· x + γ − bi |.
i=1

We can formulate a linear program to find the (x, γ ) that minimizes this measure. First,
define the matrix A and the vector b by
   
A1· b1
 A2·   b2 
   
A =  . , b =  . .
 ..   .. 
Am· bm

Next, write the linear program as follows:

min z = e y
x,γ ,y
subject to −y ≤ Ax + γ e − b ≤ y.

In this formulation, e = (1, 1, . . . , 1) ∈ Rm , so that the objective is the sum of the elements
of y. The constraints ensure that each yi is no smaller than the absolute value |Ai· x +γ −bi |,
10 Chapter 1. Introduction

while the fact that we are minimizing the sum of yi ’s ensures that each yi is chosen no larger
than it really needs to be. Hence, the minimization process chooses each yi to be equal to
|Ai· x + γ − bi |.
When n = 1 (that is, each Ai· has just a single element), this problem has a simple
geometric interpretation. Plotting Ai· on the horizontal axis and bi on the vertical axis,
this formulation finds the line in the two-dimensional (Ai· , bi ) space such that the sum of
vertical distances from the line to the data points bi is minimized.

1.3.3 Load Balancing Problem

Consider the task of balancing computational work among n processors, some of which
may already be loaded with other work. We wish to distribute the new work in such a way
that the lightest-loaded processor has as heavy a load as possible. We define the data for the
problem as follows:

pi = current load of processor i = 1, 2, . . . , n (nonnegative),

L= additional total load to be distributed,
xi = fraction of additional load L distributed to processor i, with xi ≥ 0 and
n
i=1 xi = 1,
γ = minimum of final loads after distribution of workload L.

Assuming that the new work can be distributed among multiple processors without incurring
any overhead, we can formulate the problem as follows:

max γ
x,γ
subject to γ e ≤ p + xL, e x = 1, x ≥ 0,

where e = (1, 1, . . . , 1) is the vector of 1’s with n components.

Interestingly, this is one of the few linear programs that can be solved in closed form.
When pi ≤ L/n for all i = 1, 2, . . . , n, the optimal γ is (e p + L)/n, and all processors
have the same workload γ . Otherwise, the processors that had the heaviest loads to begin
with do not receive any new work; the solution is slightly more complicated in this case
but can be determined by sorting the pi ’s. Similar solutions are obtained for the continuous
knapsack problem that we mention later.

1.3.4 Resource Allocation

Consider a company that needs to decide how to allocate its resources (for example, raw
materials, labor, or time on rented equipment) in a certain period to produce a variety of
finished products. Suppose the company is able to to produce m types of finished products
(indexed i = 1, 2, . . . , m) and that it uses n resources (indexed by j = 1, 2, . . . , n). Each
unit of finished product i yields ci dollars in revenue, whereas each unit of resource j
costs dj dollars. Suppose too that one unit of product i requires Aij units of resource j
to manufacture and that a maximum of bj units of resource j are available in this period.
The manufacturer aims to maximize their profit (defined as total revenue minus total cost)
subject to using no more resources than are available.
1.3. Applications 11

The variables in this problem are yi , i = 1, 2, . . . , m, which is the number of units of

product i, and xj , j = 1, 2, . . . , n, the number of units of resource j consumed. The linear
programming formulation is as follows:

max z = c y − d  x
x,y
subject to x = A y, x ≤ b, x, y ≥ 0.

To further explain the constraint x = A y better, we consider the j th equation of this system,
which is

xj = A1j y1 + A2j y2 + · · · + Amj ym .

Each term Aij yi indicates the amount of resource j used to manufacture the desired amount
of product i, and so the summation represents the total amount of resource j required to
make the specified amounts of the products. The bound x ≤ b ensures that we do not exceed
the available resources, and the nonnegativity constraint y ≥ 0 constrains us to produce a
nonnegative amount of each product. (The constraint x ≥ 0 is actually redundant and can
be omitted from the formulation; since all the elements of y and A are nonnegative, all
elements of x = A y must also be nonnegative.)

1.3.5 Classification
In classification problems, we are given two sets of points in the space of n dimensions Rn .
Our aim is to find a hyperplane in the space Rn that separates these two sets as accurately
as possible. We use this hyperplane to classify any new points that arise; if the new point
lies on one side of the hyperplane, we classify it as an element of the first set, while if it lies
on the other side, we place it in the second set.
Linear programming can be used to find the separating hyperplane, which is defined
by a vector w ∈ Rn and a scalar γ . Ideally, we would like each point t in the first set to
satisfy w t ≥ γ , while each point t in the second set satisfies w  t ≤ γ . To guard against
a trivial answer (note that the conditions just specified are trivially satisfied by w = 0 and
γ = 0!), we seek to enforce the stronger conditions w  t ≥ γ + 1 for points in the first
set and w t ≤ γ − 1 for points in the second set. Moreover, because the two sets may
be intermingled, it may not be able to enforce a clean separation. We define the objective
function in the linear program to be the sum of the average violations of the classification
conditions over each set.
We set up the linear program by constructing an m × n matrix M whose ith row
contains the n components of the ith points in the first set. Similarly, we construct a k × n
matrix B from the points in the second set. The violations of the condition w  t ≥ γ + 1
for points in the first set are measured by a vector y, which is defined by the inequalities
y ≥ −(Mw − γ e) + e, y ≥ 0, where e = (1, 1, . . . , 1) ∈ Rm . Similarly, violations of the
condition w t ≤ γ − 1 for points in the second set are measured by the vector z defined
by z ≥ (Bw − γ e) + e, z ≥ 0, where e ∈ Rk . In general, e will be a vector of ones of
appropriate dimension. The average violation on the first set is e y/m and on the second
12 Chapter 1. Introduction

O
O O O O
O O
x O O O O O
O O
x x x
x x O O
O
O
O x O
O
O
O M
x x O O O
x x x O x O
x x x O O
x O x x O O O
x x x
x O O O
x x x x O
x x O O O
x x x x
x x O O O
x
x x x
x O
x x x x w’x=γ+1
x x
x x
x x
B x
x
x x
x

x x
x
x
w’x=γ−1
w’x=γ

Figure 1.4. Classification using the plane w  x = γ .

set is e z/k, and so we can write the linear program as follows:
1 
min m
ey + k1 e z
w,γ ,y,z
subject to y ≥ −(Mw − γ e) + e,
z ≥ (Bw − γ e) + e,
(y, z) ≥ 0.
Figure 1.4 shows the separation in a particular example arising in breast cancer diag-
nosis (Mangasarian, Street & Wolberg (1995)). The first set M (indicated by circles in the
diagram) consists of fine needle aspirates (samples) taken from malignant tumors. Their
location in the two-dimensional space is defined by the measures of two properties of each
tumor, for example, the average cell size and the average deviation from “roundness” of the
cells in the sample. The second set B (indicated by crosses) consists of fine needle aspirates
taken from benign tumors. Note that the hyperplane w  x = γ (which in two dimensions is
simply a line) separates most of the benign points from most of the malignant points.
Another interesting application of the linear programming approach to classification
is described by Bosch & Smith (1998), who use a separating plane in three dimensions that
count the frequencies of certain words to determine that 12 disputed Federalist Papers were
probably authored by James Madison rather than Alexander Hamilton.

1.3.6 Minimum-Cost Network Flow

Network problems, which involve the optimization of a flow pattern in a network of nodes
and arcs, are important because of their applicability to many diverse practical problems.
1.3. Applications 13

4 7
1 9

10

Figure 1.5. Nodes and arcs in a network.

We consider here a particular kind of network problem known as minimum-cost network

flow, where the “flow” consists of the movement of a certain commodity along the arcs of
a network, from the nodes at which the commodity is produced to the nodes where it is
consumed. If the cost of transporting the commodity along an arc is a fixed multiple of the
amount of commodity, then the problem of minimizing the total cost can be formulated as
a linear program.
Networks, such as that depicted in Figure 1.5, consist of nodes N and arcs A, where
the arc (i, j ) connects an origin node i to a destination node j . Associated with each node i
is a divergence bi , which represents the amount of product produced or consumed at node i.
When bi > 0, node i is a supply node, while if bi < 0, it is a demand node. Associated with
each arc (i, j ) are a lower bound lij and an upper bound uij of the amount of the commodity
that can be moved along that arc. Each variable xij in the problem represents the amount of
commodity moved along the arc (i, j ). The cost of moving one unit of flow along arc (i, j )
is cij . We aim to minimize the total cost of moving the commodity from the supply nodes
to the demand nodes.
Using this notation, we can formulate the minimum-cost network flow problem as
follows:
min z= cij xij
x
(i,j )∈A
subject to xij − xj i = bi for all nodes i ∈ N ,
j :(i,j )∈A j :(j,i)∈A
lij ≤ xij ≤ uij for all arcs (i, j ) ∈ A.
The first constraint states that the net flow through each node should match its divergence.
The first summation represents the total flow out of node i, summed over all the arcs that have
node i as their origin. The second summation represents total flow into node i, summed over
14 Chapter 1. Introduction

all the arcs having node i as their destination. The difference between inflow and outflow
is constrained to be the divergence bi .
By relabeling the flow variables as x1 , x2 , . . . , xn , where n is the total number of
arcs, we can put the problem into a more general programming form. However, the special
notation used above reveals the structure of this application, which can be used in designing
especially efficient versions of the simplex method. Note, in particular, that the coefficient
matrix arising from the flow constraints contains only the numbers 0, 1, and −1. If all
the problem data is integral, it can be shown that the solution x also contains only integer
components.

1.4 Algorithms and Complexity

Though easy to state, linear programs can be quite challenging to solve computationally.
The essential difficulty lies in determining which of the inequality constraints and bounds are
active (that is, satisfied as equalities) at the solution and which are satisfied but inactive. (For
example, the constraint 2x1 + x2 ≤ 8 is active at the point (x1 , x2 ) = (1, 6); it is satisfied but
inactive at the point (2, 2); it is violated at the point (4, 1).) To determine which constraints
are active at the solution would seem to be a combinatorial problem: If there are l inequality
constraints and bounds, and each of them can be either active or inactive, we may have a
total of 2l active/inactive combinations. The situation hardly improves if we make use of
the fact that a solution occurs at one of the vertices of the feasible region, defined as a point
at which at least n of the constraints are active. A problem in Rn with a total of l inequality
constraints and bounds (and no equality constraints) may have as many as
l l!
=
n (l − n)!n!
vertices. Even for a small problem with n = 10 variables and l = 20 inequality constraints
and bounds, there may be 184, 756 vertices, and possibly 1, 048, 576 active/inactive com-
binations. A “brute force” algorithm that examines all these possibilities will be much too
slow for practical purposes.

1.4.1 The Simplex Method

From a geometrical point of view, the simplex method is easy to understand. It starts
by determining whether the feasible region is empty. If so, it declares the problem to be
infeasible and terminates. Otherwise, it finds a vertex of the feasible region to use as a
starting point. It then moves from this vertex to an adjacent vertex for which the value of
the objective z is lower—in effect, sliding along an edge of the feasible region until it can
proceed no further without violating one of the constraints. This process is repeated; the
algorithm moves from vertex to (adjacent) vertex, decreasing z each time. The algorithm
can terminate in one of two ways. First, it may encounter a vertex whose value of z is less
than or equal to all adjacent vertices. In this case, it declares this vertex to be a solution of
the linear program. Second, it may detect that the problem is unbounded. That is, it may
find a direction leading away from the current vertex that remains feasible (no matter how
long a step is taken along it) such that the objective z decreases to −∞ along this direction.
In this case, it declares the problem to be unbounded.
1.4. Algorithms and Complexity 15

Suppose in our two-variable example of Figure 1.1 that the simplex algorithm starts at
the origin (0, 0). It could find the optimum (4, 5) by moving along one of two paths where,
due to conversion to a minimization problem, z is the negative of that depicted in Figure 1.1.

Path 1 Path 2
(0,0) z=0
(0,0) z=0
(6,0) z = −30
(0,6) z = −24
(6,2) z = −38
(4,5) z = −40
(4,5) z = −40

Note that both adjacent vertices of the initial point (0, 0) have lower objective values,
and hence each one is a valid choice for the next iterate. The simplex method uses a pivot
selection rule to select from among these possibilities; different variants of the simplex
method use different pivot rules, as we see in Chapters 3 and 5.

1.4.2 Interior-Point Methods

Although the simplex method performs well on most practical problems, there are patho-
logical examples (Klee & Minty (1972)) in which the number of iterations required is
exponential in the number of variables. On such examples, linear programming seems to
reveal a combinatorial nature. A surprising development occurred in 1979, when a (theoret-
ically) more efficient method was discovered by Khachiyan (1979). For problems in which
the data A, b, c were integer or rational numbers, Khachiyan’s ellipsoid method can solve the
problem in a time that is bounded by a polynomial function of the number of bits L needed
to store the data and the number of variables n. However, the ellipsoid method proved to
be difficult to implement and disappointingly slow in practice. Karmarkar (1984) proposed
a new algorithm with a similar polynomial bound. He made the additional claim that a
computational implementation of his algorithm solved large problems faster than existing
simplex codes. Though this claim was never fully borne out, Karmarkar’s announcement
started a surge of new research into interior-point methods, so named because their iterates
move through the interior of the feasible region toward a solution, rather than traveling
from vertex to vertex around the boundary. Software based on interior-point methods is
often significantly faster than simplex codes on large practical problems. We discuss these
methods further in Chapter 8.

Notes and References

The use of the word “programming” in connection with linear programming is some-
what anachronistic. It refers to the step-by-step mathematical procedure used to solve this
optimization problem, not specifically to its implementation in a computer program. The
term “linear programming” was coined in the 1940s, well before the word “programming”
became strongly associated with computers.
The definition of the term standard form is itself not “standard”; other authors use
a definition different from the one we provide in (1.2). The term canonical form is not
widely used and is also not standard terminology, but we use it here as a convenient way to
distinguish between the two formulations, both of which appear throughout the book.
16 Chapter 1. Introduction

The classic text on the simplex method is by the inventor of this method, George
B. Dantzig (1963). In 1939, the Russian Nobel Laureate Leonid V. Kantorovich had also
proposed a method for solving linear programs; see Kantorovich (1960).
More advanced treatments of linear programming than ours include the books of
Chvátal (1983) and Vanderbei (1997). Wright (1997) focuses on interior-point methods.
Several advanced chapters on linear programming (both simplex and interior-point) also
appear in the text of Nocedal & Wright (2006). The latter text also contains material on
more general optimization problems, especially nonlinear optimization problems with and
without constraints. The text of Wolsey (1998) provides an excellent introduction to integer
programming.
Chapter 2

Linear Algebra:
A Constructive Approach

In Section 1.4 we sketched a geometric interpretation of the simplex method. In this chapter,
we describe the basis of an algebraic interpretation that allows it to be implemented on a
computer. The fundamental building block for the simplex method from linear algebra is the
Jordan exchange. In this chapter, we describe the Jordan exchange and its implementation
in MATLAB. We use it in a constructive derivation of several key results in linear algebra
concerning linear independence and the solution of linear systems of equations. In the
latter part of the chapter, we discuss the LU factorization, another linear algebra tool that
is important in implementations of the simplex method.
In this chapter and the rest of the book, we assume basic familiarity with MATLAB.
There are many books and web sites that will get you started in MATLAB; we recommend
the MATLAB Primer by Sigmon & Davis (2004).
We first describe the Jordan exchange, a fundamental building block of linear algebra
and the simplex algorithm for linear programming.

2.1 Jordan Exchange

Consider the following simple linear equation in the one-dimensional variables x and y:

y = ax.

The form of the equation indicates that x is the independent variable and y is the dependent
variable: Given a value of x, the equation tells us how to determine the corresponding value
of y. Thus we can think of the dependent variable as a function of the independent variable;
that is, y(x) := ax. If we assume that a  = 0, we can reverse the roles of y and x as follows:

1
x = ãy, where ã = .
a
Note that now we have a function x(y) in which x is determined as a function of y. This
exchange in roles between dependent and independent variables gives a very simple proce-
dure for solving either the equation ax − b = 0 or the inequality ax − b ≥ 0, using the

17
18 Chapter 2. Linear Algebra: A Constructive Approach

following simple equivalences:

ax − b = 0 ⇐⇒ ax = y, y=b ⇐⇒ x = ãy, y = b,
(2.1)
ax − b ≥ 0 ⇐⇒ ax = y, y≥b ⇐⇒ x = ãy, y ≥ b.

In particular, the second formula gives an explicit characterization of the values of x that
satisfy the inequality ax − b ≥ 0, in terms of an independent variable y for which y ≥ b.
The Jordan exchange is a generalization of the process above. It deals with the
case in which x ∈ Rn is a vector of independent variables and y ∈ Rm is a vector of
dependent variables, and we wish to exchange one of the independent variables with one
of the dependent variables. The Jordan exchange plays a crucial role in solving systems
of equations, linear inequalities, and linear programs. In addition, it can be used to derive
fundamental results of linear algebra and linear programming.
We now demonstrate a Jordan exchange on the system y = Ax by exchanging the roles
of a component yr of y and a component xs of x. First, we write this system equation-wise as

yi = Ai1 x1 + Ai2 x2 + · · · + Ain xn , i = 1, 2, . . . , m, (2.2)

where the independent variables are x1 , x2 , . . . , xn and the dependent variables are
y1 , y2 , . . . , ym , and the Aij ’s are the coefficients. We can think of the yi ’s as (linear)
functions of xj ’s, that is,

yi (x) := Ai1 x1 + Ai2 x2 + · · · + Ain xn , i = 1, 2, . . . , m, (2.3)

or, more succinctly, y(x) := Ax. This system can also be represented in the following
tableau form:
x1 · · · xs · · · xn
y1 = A11 · · · A1s · · · A1n
.. .. .. ..
. . . .
yr = Ar1 · · · Ars · · · Arn
.. .. .. ..
. . . .
ym = Am1 ··· Ams ··· Amn
Note that the tableau is nothing more than a compact representation of the system of equa-
tions (2.2) or the functions determining the dependent variables from the independent vari-
ables (2.3). All the operations that we perform on the tableau are just simple algebraic
operations on the system of equations, rewritten to conform with the tableau representation.
We now describe the Jordan exchange or pivot operation with regard to the tableau
representation. The dependent variable yr will become independent, while xs changes from
being independent to being dependent. The process is carried out by the following three
steps.
(a) Solve the rth equation

yr = Ar1 x1 + · · · + Ars xs + · · · + Arn xn

for xs in terms of x1 , x2 , . . . , xs−1 , yr , xs+1 , . . . , xn . Note that this is possible if and

only if Ars  = 0. (Ars is known as the pivot element, or simply the pivot.)
2.1. Jordan Exchange 19

(b) Substitute for xs in all the remaining equations.

(c) Write the new system in a new tableau form as follows:

x1 ··· xs−1 yr xs+1 ··· xn

y1 = B11 ··· B1s ··· B1n
.. .. .. ..
. . . .
yr−1 =
xs = Br1 ··· Brs ··· Brn
yr+1 =
.. .. .. ..
. . . .
ym = Bm1 ··· Bms ··· Bmn

To determine the elements Bij in terms of the elements Aij , let us carry out the algebra
specified by the Jordan exchange. As will be our custom in this book, we will describe
and produce corresponding MATLAB m-files for the important algebraic operations that
we perform. Solution of the rth equation

n
yr = Arj xj + Ars xs
j =1
j =s

for xs gives
n n
1 −Arj
xs = yr + xj = Brs yr + Brj xj , (2.4)
Ars j =1
Ars j =1
j =s j =s

where
1 −Arj
Brs = , Brj = ∀j  = s. (2.5)
Ars Ars

These formulae define the rth row Br· of the transformed tableau. We can express them
in MATLAB by first defining J to represent the columns of the tableau excluding the sth
column, that is,

J = [1:s-1,s+1:n];

and then writing

B(r,s) = 1.0/A(r,s); B(r,J) = -A(r,J)/A(r,s);

This “vector index” facility is an important feature of MATLAB which enables terse coding
of expressions such as those given in (2.5).
20 Chapter 2. Linear Algebra: A Constructive Approach

We can now proceed with part (b) of the Jordan exchange. Substituting of the expres-
sion for xs from (2.4) in the ith equation of the tableau (i  = r), we have
 
n
 1 n
−Arj 
yi = Aij xj + Ais 
 Ars y r + xj 

j =1 j =1
A rs
j  =s j =s
n
= Bij xj + Bis yr ,
j =1
j  =s

where
Ais Ais  
Bis = , Bij = Aij − Arj = Aij − Bis Arj ∀i  = r, j  = s. (2.6)
Ars Ars

These formulae define rows Bi· , i = 1, 2, . . . , m, i  = r, of the transformed tableau. We

can also write these equations succinctly in MATLAB notation by defining J as above and
defining I to represent all the rows of the tableau except the rth row, that is,

I = [1:r-1,r+1:m];

and writing

B(I,s) = A(I,s)/A(r,s);
B(I,J) = A(I,J) - B(I,s)*A(r,J);

The complete description of one step of the Jordan exchange with pivot Ars is coded
in jx.m—the function jx in MATLAB.
Note that we have introduced the “function” facility of MATLAB. Any function can be
defined in a file of the same name as the function, but with suffix .m, just as the function jx
is stored in jx.m. It can then be invoked from within MATLAB—either from the command
window or from within other functions—by simply typing the function name together with
its arguments. The following example shows a call to the function jx.

Example 2-1-1. Solve the following system of equations for x1 , x2 in terms of y1 , y2 :

y1 = 2x1 + x2 ,
y2 = 3x1 + x2 .

Working from the MATLAB command window, one first loads the data file containing
the matrix A of Example 2-1-1 and then invokes jx twice to perform the two required
Jordan exchanges:

load ex2-1-1
B = jx(A,1,1)
B = jx(B,2,2)
2.1. Jordan Exchange 21

MATLAB file jx.m: Jordan exchange

function B = jx(A,r,s)
% syntax: B = jx(A,r,s)
% input: matrix A, integers r,s
% perform a Jordan exchange with pivot A(r,s)

[m,n] = size(A); B = zeros(m,n);

I = [1:r-1,r+1:m]; J = [1:s-1,s+1:n];

% update pivot row

B(r,s) = 1.0/A(r,s); B(r,J) = -A(r,J)/A(r,s);

% update pivot column

B(I,s) = A(I,s)/A(r,s);

% update remainder of tableau

B(I,J) = A(I,J)-B(I,s)*A(r,J);
return;

Note that we overwrite B at each step to hold the cumulative effects of the sequence of
exchanges.
We now introduce a MATLAB structure that stores a complete tableau—the row and
column labels corresponding to the dependent and independent variables, along with the
matrix of coefficients that defines the relationship between these quantities. The totbl
command can be used to construct a tableau as follows:

load ex2-1-1 x1 x2
T = totbl(A); y1 = 2 1
y2 = 3 1

The row labels (dependent variables) are assigned the default values y1 and y2 and the
column labels (independent variables) the default values x1 and x2 ; other forms of the
totbl command, discussed later, will allow the user to define their own labels. The
command tbl can be used to print out the tableau along with its associated labels.
To perform Jordan exchanges on the tableau representation (rather than on just the
matrix), we use the labeled Jordan exchange function ljx in place of jx as follows:

T = ljx(T,1,1); y1 x2
x1 = 0.5 −0.5
y2 = 1.5 −0.5
22 Chapter 2. Linear Algebra: A Constructive Approach

T = ljx(T,2,2); y 1 y2
x1 = −1 1
x2 = 3 −2

In addition to making the algebraic changes to the matrix, ljx swaps the row and column
labels as required by the exchange and prints the modified tableau using tbl. The trailing
semicolon should not be omitted after the call to ljx since it results in the printing of
additional unnecessary information about the structure T.

The following simple theorem provides a formal justification of the Jordan exchange
formulae (2.5) and (2.6) as well as their extension to multiple pivots in succession. The
result will enable us to use the Jordan exchange to give some constructive derivations of key
results in linear algebra and linear programming.

Theorem 2.1.1. Consider the linear function y defined by y(x) := Ax, where A ∈ Rm×n .
After k pivots (with appropriate reordering of rows and columns) denote the initial and kth
tableaus as follows:
xJ1 xJ2 yI1 xJ2
yI1 = AI1 J1 AI1 J2 xJ1 = BI1 J1 BI1 J2
yI2 = AI2 J1 AI2 J2 yI2 = BI2 J1 BI2 J2

Here I1 , I2 is a partition of {1, 2, . . . , m} and J1 , J2 is a partition of {1, 2, . . . , n}, with I1

and J1 containing the same number of elements. Then for all values of x ∈ Rn

xJ1 = BI1 J1 yI1 (x) + BI1 J2 xJ2 ,

yI2 (x) = BI2 J1 yI1 (x) + BI2 J2 xJ2 .

That is, the original linear functions y satisfy the new linear relationships given by the kth
tableau.

Proof. We show the result for one pivot. The result for k pivots follows by induction.
For a pivot on the (r, s) element, we have I1 = {r}, I2 = {1, . . . , r − 1, r + 1, . . . , m},
J1 = {s}, and J2 = {1, . . . , s − 1, s + 1, . . . , n}. Then for all x, we have

n
xJ1 − BI1 J1 yI1 (x) − BI1 J2 xJ2 = xs − Brs yr (x) − Brj xj
j =1
j =s
 
n n
1  −Arj
= xs − Arj xj  − xj
Ars j =1 j =1
Ars
j =s
=0
2.2. Linear Independence 23

and

yI2 (x) − BI2 J1 yI1 (x) − BI2 J2 xJ2

 
 n
 i = 1, 2, . . . , m,
=
yi (x) − Bis yr (x) − Bij xj 
, i = r
j =1
j  =s

 
 n
Ais
n n
Ais Arj  i = 1, 2, . . . , m,
=
 Aij xj − Arj xj − Aij − xj 
,
Ars Ars i = r
j =1 j =1 j =1
j =s
= 0,

verifying the claims.

The following result shows that if two tableaus have identical dependent variables for
all possible values of the independent variables, then the tableau entries are also identical.

Proposition 2.1.2. If the linear function y is defined by y(x) = Ax and also by y(x) = Bx,
then A = B.

Proof. Since (A − B)x = 0 for all x ∈ Rn , we can choose x = I·i , where I·i is the ith
column of the identity matrix. We deduce that the ith columns of A and B are identical.
Since this fact is true for all i = 1, 2, . . . , n, we conclude that A = B.

2.2 Linear Independence

A simple geometric way to solve a system of two equations in two unknowns is to plot the
corresponding lines and determine the point where they intersect. Of course, this technique
fails when the lines are parallel to one another. A key idea in linear algebra is that of
linear dependence, which is a generalization of the idea of parallel lines. Given a matrix
A ∈ Rm×n , we may ask if any of its rows are redundant. In other words, is there a row Ak·
that can be expressed as a linear combination of the other rows? That is,
m
Ak· = λi Ai· . (2.7)
i=1
i =k

If so, then the rows of A are said to be linearly dependent.

As a concrete illustration, consider the matrix
 
1 2 4
A = 3 4 8  .
5 6 12
24 Chapter 2. Linear Algebra: A Constructive Approach

The third row of this matrix is redundant, because it can be expressed as a linear combination
of the first two rows as follows:

A3· = 2A2· − A1· .

If we rearrange this equation, we see that

 
−1 2 −1 A = 0,
 
that is, for z = −1 2 −1 , z A = 0 with z  = 0.
We define linear dependence of the rows of a matrix A formally as follows:

z A = 0 for some nonzero z ∈ Rm .

(We see that (2.7) can be expressed in this form by taking zi = λi , i  = k, zk = −1.) The
negation of linear dependence is linear independence of the rows of A, which is defined by
the implication
z A = 0 ⇒ z = 0.
The idea of linear independence extends also to functions, including the linear func-
tions y defined by y(x) := Ax that we have been considering above. The functions yi (x),
i = 1, 2, . . . , m, defined by y(x) := Ax are said to be linearly dependent if

z y(x) = 0 ∀x ∈ Rn for some nonzero z ∈ Rm

and linearly independent if

z y(x) = 0 ∀x ∈ Rn ⇒ z = 0. (2.8)

The equivalence of the linear independence definitions for matrices and functions is clear
when we note that

z Ax = 0 ∀x ∈ Rn for some nonzero z ∈ Rm

⇐⇒ z A = 0 for some nonzero z ∈ Rm .

Thus the functions y(x) are linearly independent if and only if the rows of the matrix A are
linearly independent.
Proposition 2.2.1. If the m linear functions yi are linearly independent, then any p of them
are also linearly independent, where p ≤ m.

Proof. The proof is obvious from contrapositive statement: yi , i = 1, 2, . . . , p, are linearly

dependent implies yi , i = 1, 2, . . . , m, are linearly dependent.

Proposition 2.2.2. If the linear functions y defined by y(x) = Ax, A ∈ Rm×n , are linearly
independent, then m ≤ n. Furthermore, in the tableau representation, all m dependent yi ’s
can be made independent; that is, they can be exchanged with m independent xj ’s.
2.2. Linear Independence 25

Proof. Suppose that the linear functions y(x) = Ax are linearly independent. Exchange
y’s and x’s in the tableau until no further pivots are possible, at which point we are blocked
by a tableau of the following form (after a possible rearrangement of rows and columns):

yI1 xJ2
xJ1 = BI1 J1 BI1 J2
yI2 = BI2 J1 0

If I2  = ∅, we have by Theorem 2.1.1 that

yI2 (x) = BI2 J1 yI1 (x) ∀x ∈ Rn ,

which we can rewrite as follows:

 
  yI1 (x)
−BI2 J1 I = 0.
yI2 (x)
 
By taking z to be any row of the matrix −BI2 J1 I , note that z is nonzero and that
z y(x) = 0. According to definition (2.8), the existence of z implies that the functions
y(x) = Ax are linearly dependent. Hence, we must have I2 = ∅, and therefore m ≤ n and
all the yi ’s have been pivoted to the top of the tableau, as required.

Example 2-2-1. For illustration, consider the matrix defined earlier:

 
1 2 4
A = 3 4 8  .
5 6 12

Then

load ex2-2-1 x1 x2 x3
T = totbl(A); y1 = 1 2 4
y2 = 3 4 8
y3 = 5 6 12

T = ljx(T,1,1); y1 x2 x3
x1 = 1 −2 −4
y2 = 3 −2 −4
y3 = 5 −4 −8

T = ljx(T,2,2); y1 y2 x3
x1 = −2 1 0
x2 = 1.5 −0.5 −2
y3 = −1 2 0
26 Chapter 2. Linear Algebra: A Constructive Approach

Note that we cannot pivot any more y’s to the top and that y3 = −y1 + 2y2 . This final
relationship indicates the linear dependence relationship amongst the rows of A, namely
that A3· = −A1· + 2A2· .
The above result can be strengthened to the following fundamental theorem, which
can be taken as an alternative and constructive definition of linear independence.
Theorem 2.2.3 (Steinitz). For a given matrix A ∈ Rm×n , the linear functions y, defined by
y(x) := Ax, are linearly independent if and only if for the corresponding tableau all the
yi ’s can be exchanged with some m independent xj ’s.

Proof. The “only if” part follows from Proposition 2.2.2, and so we need to prove just
the “if” part. If all the yi ’s can be exchanged to the top of the tableau, then we have (by
rearranging rows and columns if necessary) that
xJ1 xJ2 y xJ2
y = A·J1 A·J2 −→ xJ1 = B·J1 B·J2

Suppose now that there is some z such that z A = 0. We therefore have that z Ax = 0 for
all x ∈ Rn . In the right-hand tableau above, we may set the independent variables y = z,
xJ2 = 0, whereupon xJ1 = B·J1 z. For this particular choice of x and y, we have y = Ax
from Theorem 2.1.1, and so it follows that

0 = z Ax = z y = z z,

implying that z = 0. We have shown that the only z for which z A = 0 is the zero vector
z = 0, verifying that the rows of A and hence the functions y are linearly independent.

A consequence of this result is that given a matrix A, the number of linearly indepen-
dent rows in A is the maximum number of components of y that can be exchanged to the
top of the tableau for the functions y(x) := Ax.
When not all the rows of A are linearly independent, we reach a tableau in which one
or more of the yi ’s are expressed in terms of other components of y. These relationships
show the linear dependencies between the functions y(x) and, therefore, between the rows
of the matrix A.
Example 2-2-2. Let the matrix A be defined by
 
−1 0 3
A=  2 −2 4 .
0 −2 10
By using ljx.m, find out how many linearly independent rows A has. If there are linear
dependencies, write them out explicitly.
After entering the matrix A into MATLAB, we construct a tableau and perform two
Jordan exchanges to make y1 and y2 independent variables:
T=totbl(A); x 1 x2 x3
y1 = −1 0 3
y2 = 2 −2 4
y3 = 0 −2 10
2.3. Matrix Inversion 27

T=ljx(T,2,1); y2 y1 x3
T=ljx(T,1,2); x2 = −0.5 −1 5
x1 = 0 −1 3
y3 = 1 2 0

We cannot exchange y3 with x3 because there is a zero in the pivot position. We

conclude that this matrix has two linearly independent rows. By reading across the final
row of the tableau, we see that the components of y are related as follows:

y3 = y2 + 2y1 .

In this example, we could have done the Jordan exchanges in some other way; for
example, T=ljx(T,1,1) followed by T=ljx(T,3,3). However, the relationship that
we derive between the components of y will be equivalent.
Exercise 2-2-3. Let  
1 2 3 4
A= 3 1 3 0 .
1 3 −3 −8
Using ljx.m, find out how many linearly independent rows A has. By working with A ,
find out how many linearly independent columns it has. If there are linear dependencies,
write them out explicitly.

2.3 Matrix Inversion

An n × n matrix is nonsingular if the rows of A are linearly independent; otherwise the
matrix is singular.
Theorem 2.3.1. The system y = Ax with A ∈ Rn×n can be inverted to x = By if and only
if A is nonsingular. In this case, the matrix B is unique and is called the inverse of A and
is denoted by A−1 . It satisfies AA−1 = A−1 A = I .

Proof. Apply Steinitz’s theorem (Theorem 2.2.3) to A to get B such that x = By. B is
unique by Proposition 2.1.2. Finally, y = Ax = ABy for all y shows that I − AB = 0,
and x = By = BAx for all x shows that I − BA = 0.

Example 2-3-1. Invert the matrix

 
2 −1 −1
A = 0 2 −1
0 −1 1
using MATLAB and the ljx function.
load ex2-3-1 x1 x2 x3
T = totbl(A); y1 = 2 −1 −1
y2 = 0 2 −1
y3 = 0 −1 1
28 Chapter 2. Linear Algebra: A Constructive Approach

T = ljx(T,1,1); y1 x2 x3
x1 = 0.5 0.5 0.5
y2 = 0 2 −1
y3 = 0 −1 1

T = ljx(T,2,2); y1 y2 x3
x1 = 0.5 0.25 0.75
x2 = 0 0.5 0.5
y3 = 0 −0.5 0.5

T = ljx(T,3,3); y1 y2 y3
x1 = 0.5 1 1.5
x2 = 0 1 1
x3 = 0 1 2

Note that the inverse of A is now found in this tableau, that is,
 
0.5 1 1.5
invA = T.val A−1 =  0 1 1
0 1 2

The command T.val extracts the matrix from the tableau T (that is, it strips off the labels).
We can check that the computed matrix is indeed A−1 by evaluating AA−1 − I using
the following code:

A*invA-eye(3)

(where eye(3) is MATLAB notation for a 3 × 3 identity matrix). This code should result
in a 3 × 3 matrix whose elements are zero or very small numbers (which may not be exactly
zero because of numerical roundoff error).
As an alternative method for solving this problem, we show that it is possible not to
pivot along the diagonal. For the matrix B of Theorem 2.3.1 to be A−1 , the subscript indices
of x and y must be both arranged in ascending order. If this is not the case, reordering of
rows and/or columns is necessary.

load ex2-3-1 x1 x2 y1
T = totbl(A); x3 = 2 −1 −1
y2 = −2 3 1
T = ljx(T,1,3); y3 = 2 −2 −1
2.3. Matrix Inversion 29

T = ljx(T,3,1); y3 x2 y1
x3 = 1 1 0
y2 = −1 1 0
x1 = 0.5 1 0.5

T = ljx(T,2,2); y 3 y2 y1
x3 = 2 1 0
x2 = 1 1 0
x1 = 1.5 1 0.5

Notice that the numbers in this final tableau are identical to those obtained after the previous
sequence of pivots but that the rows and columns have been reordered according to the
labels. To restore the correct ordering of the rows and columns and recover the inverse A−1 ,
we use the command

Atemp = T.val;

to extract the matrix from the tableau and then use standard MATLAB commands to reorder
the rows and columns of the 3 × 3 matrix Atemp to obtain A−1 . In this case, we note from
the row labels that rows 1, 2, and 3 of T must appear as rows 3, 2, and 1 of A−1 , respectively;
while from the column labels we see that columns 1, 2, and 3 of T must appear as rows
3, 2, and 1 of A−1 , respectively. We can define permutation vectors I and J to effect the
reordering and then define A−1 as follows:

I=[3 2 1]; J=[3 2 1];

invA(I,J)=Atemp;

Alternatively, we can avoid generating Atemp and instead use

invA(I,J)=T.val;

Note that the permutations correspond to the row and column label orderings after the final
Jordan exchange. That is, the vector I shows the final ordering of the row labels (x3 , x2 ,
x1 ), and the vector J shows the final ordering of the column labels (y3 , y2 , y1 ). This scheme
for choosing the reordering vectors will work in general, provided we put the reordering on
the left-hand side of the assignment of T.val to invA, as above.

Of course, for the example above, one can avoid the final reordering step by pivoting
down the diagonal in order. However, there may be problems for which such a pivot sequence
is not possible. A simple example is given by the matrix
 
0 1
A= ,
1 1

for which the (1, 1) element cannot be used as the first pivot. Another example follows.
30 Chapter 2. Linear Algebra: A Constructive Approach

Example 2-3-2. Calculate A−1 using ljx, where

 
0 1 3
A = 4 3 2  .
1 6 6

load ex2-3-2 x1 x2 x3
T = totbl(A); y1 = 0 1 3
y2 = 4 3 2
y3 = 1 6 6

T = ljx(T,3,1); y3 x2 x3
y1 = 0 1 3
y2 = 4 −21 −22
x1 = 1 −6 −6

T = ljx(T,1,2); y3 y1 x3
x2 = 0 1 −3
y2 = 4 −21 41
x1 = 1 −6 12

T = ljx(T,2,3); y3 y1 y2
x2 = 0.2927 −0.5366 −0.0732
x3 = −0.0976 0.5122 0.0244
x1 = −0.1707 0.1463 0.2927

We now extract the numerical values from the tableau, define the permutation vectors, and
perform the reordering as follows:
 
0.1463 0.2927 −0.1707
I=[2 3 1]; J=[3 1 2]; A−1 = −0.5366 −0.0732 0.2927 
invA(I,J) = T.val; 0.5122 0.0244 −0.0976

Exercise 2-3-3. Calculate A−1 using ljx.m, where

 
1 2 3
A = 2 4 2 .
1 1 1
2.3. Matrix Inversion 31

Exercise 2-3-4. Use ljx.m to find the inverses of the following matrices in MATLAB:
     
1 0 1 1 0 1 2 0 1
A = 2 1 1 , B = 2 1 1 , C = 1 2 0.5 .
3 0 0 1 1 0 0 1 1
If a matrix is singular, show the linear dependence between the rows of the matrix. (Use
T.val and perform any reordering needed on the resulting MATLAB matrix to obtain the
final result.)
At this point, we note that the pivot rules described in Section 2.1 in terms of matrix
elements have simple matrix block analogues. That is, instead of pivoting on Ars , we could
instead pivot on the submatrix ARS , where R ⊆ {1, 2, . . . , m} and S ⊆ {1, 2, . . . , n} are two
index sets with the same number of elements. The only real difference is that the inverse
of ARS is used in place of 1/Ars . MATLAB code for the block Jordan exchange is given
in bjx.m. The code incorporates changes to the labels, and so it is an extension of ljx
rather than of jx. In addition, the code updates the matrix A instead of creating the new
matrix B, and so it needs to perform the operations in a slightly different order from the
analogous operations in ljx.
Two successive Jordan exchanges can be effected as one block pivot of order two.
Thus by induction, any sequence of Jordan exchanges can be effected by a single block
pivot. What are the algebraic consequences of this observation? Consider a system that
leads to the following tableau:
(2.9)
A B
C D
where A is square and invertible. Applying a block pivot to the matrix A, the transformed
tableau is
(2.10)
A−1 −A−1 B
CA−1 D − CA−1 B
 B
The matrix D − CA−1 B is called the Schur complement of A in CA D . The algebraic
formula for the block pivot operation will prove to be very useful in what follows. For
example, if A−1 exists, then the original matrix (2.9) is invertible if and only if the Schur
complement D − CA−1 B is invertible.
Exercise 2-3-5. The following tableau expresses how the dependent variables yI1 and yI2
can be expressed in terms of the independent variables xJ1 and xJ2 :
xJ1 xJ2
yI1 = A B
yI2 = C D
By performing some simple manipulations, derive the following formulae:
xJ1 = A−1 yI1 − A−1 BxJ2 and yI2 = CA−1 yI1 + (D − CA−1 B)xJ2 ,
thus justifying the block Jordan exchange formula (2.10).
32 Chapter 2. Linear Algebra: A Constructive Approach

MATLAB file bjx.m: Labeled block Jordan exchange

function A = bjx(A,R,S)
% syntax: B = bjx(A,R,S)
% input: tableau A, integer vectors R,S
% perform a block Jordan exchange with pivot A(R,S)

R = R(:); S = S(:);
[m,n] = size(A.val);

% setdiff(1:m,R) := {1,...,m}\R
I = setdiff(1:m,R); J = setdiff(1:n,S);

% note that values are updated in place

% update pivot column
A.val(R,S) = inv(A.val(R,S));
A.val(I,S) = A.val(I,S)*A.val(R,S);

% update remainder of tableau

A.val(I,J) = A.val(I,J)-A.val(I,S)*A.val(R,J);

% update pivot row

A.val(R,J) = -A.val(R,S)*A.val(R,J);

% now update the labels

swap = A.bas(R);
A.bas(R) = A.nonbas(S);
A.nonbas(S) = swap;

if isfield(A,’dualbas’)
swap = A.dualbas(S);
A.dualbas(S) = A.dualnonbas(R);
A.dualnonbas(S) = swap;
end

tbl(A);

return;

2.4 Exact Solution of m Equations in n Unknowns

At this stage, we have considered only square systems where the number of variables is
the same as the number of equations. In optimization applications, it is more likely that
the systems under consideration have different numbers of variables than equations. Such
2.4. Exact Solution of m Equations in n Unknowns 33

systems pose additional concerns. For example, the system

x1 + x2 = 1

having one equation and two variables has infinitely many solutions, whereas the system

x1 = 1, x1 = 2

clearly has no solution.

Suppose we wish to solve Ax = b (or determine that no solution exists), where
A ∈ Rm×n and b ∈ Rm , with m and n not necessarily equal. The Jordan exchange gives a
simple method for solving this problem under no assumption whatsoever on A.
1. Write the system in the following tableau form:

x 1
y = A −b

Our aim is to seek x and y related by this tableau such that y = 0.

2. Pivot as many of the yi ’s to the top of the tableau, say yI1 , until no more can be
pivoted, in which case we are blocked by a tableau as follows (with row and column
reordering):
yI1 xJ2 1
xJ1 = BI1 J1 BI1 J2 dI1
yI2 = BI2 J1 0 dI2
We now ask the question: Is it possible to find x and y related by this tableau such
that y = 0?
3. The system is solvable if and only if dI2 = 0, since we require yI1 = 0 and yI2 = 0.
When dI2 = 0, we obtain by writing out the relationships in the tableau explicitly that

yI1 = 0,
yI2 = BI2 J1 yI1 = 0,
xJ2 is arbitrary,
xJ1 = BI1 J2 xJ2 + dI1 .

Note that m could be less than or greater than n.

Example 2-4-1. Solve the following system:

x1 − x2 + x3 = 2,
−x1 + 2x2 + x3 = 3,
x1 − x2 − x3 = −2.
Remember that our technique is to pivot as many of the yi ’s to the top of the tableau
as possible and then check that the resulting tableau is still valid. In particular, the elements
in the last column corresponding to the yi ’s that we cannot pivot to the top—the subvector
dI2 in the notation above—should be zero.
34 Chapter 2. Linear Algebra: A Constructive Approach

Rewrite the problem as follows:

y1 = x1 − x2 + x3 − 2,
y2 = −x1 + 2x2 + x3 − 3,
y3 = x1 − x2 − x3 + 2,

and carry out the following pivot operations on its tableau representation:

load ex2-4-1 x1 x2 x3 1
T = totbl(A,b); y1 = 1 −1 1 −2
y2 = −1 2 1 −3
y3 = 1 −1 −1 2

This gives rise to the following sequence of tableaus:

T = ljx(T,1,1); y1 x2 x3 1
x1 = 1 1 −1 2
y2 = −1 1 2 −5
y3 = 1 0 −2 4

T = ljx(T,2,2); y1 y2 x3 1
x1 = 2 1 −3 7
x2 = 1 1 −2 5
y3 = 1 0 −2 4

T = ljx(T,3,3); y1 y2 y3 1
x1 = 0.5 1 1.5 1
x2 = 0 1 1 1
x3 = 0.5 0 −0.5 2

The final solution can be read off the tableau by setting y1 = y2 = y3 = 0. We get x1 = 1,
x2 = 1, and x3 = 2. Note that if the calculations are being carried out by hand, the columns
of the tableau that are labeled with a yi can be suppressed since their values are never needed.
This can result in a significant saving in computation time, particularly if you are performing
the steps by hand.

load ex2-4-1 x2 x3 1
T = ljx(T,1,1); x1 = 1 −1 2
y2 = 1 2 −5
T = delcol(T,’y1’); y3 = 0 −2 4
2.4. Exact Solution of m Equations in n Unknowns 35

T = ljx(T,2,1); x3 1
T = delcol(T,’y2’); x1 = −3 7
x2 = −2 5
y3 = −2 4

T = ljx(T,3,1); 1
T = delcol(T,’y3’); x1 = 1
x2 = 1
x3 = 2

Of course, by deleting the yi columns, we lose access to the linear dependence relationships
between these variables.

Exercise 2-4-2. Test yourself on the following example:

x1 + x2 + x3 = 1,
x1 − x2 − x3 = 1,
x1 − x2 + x3 = 3,

which has solution x = (1, −1, 1) using

load ex2-4-2
T = totbl(A,b);
...

Exercise 2-4-3. Solve the following systems of equations:

1.
2u + 3v + 3w = 2,
5v + 7w = 2,
6u + 9v + 8w = 5.

2.
u + 4v + 2w = −2,
−2u − 8v + 3w = 32,
v + w = 1.

Example 2-4-4. Solve the following system of 3 equations in 4 unknowns:

x1 − x2 + x4 = 1,
x1 + x3 = 1,
x1 + x2 + 2x3 − x4 = 0.

The data file ex2-4-4.mat can be loaded into MATLAB enabling the following sequence
of tableaus to be constructed:
36 Chapter 2. Linear Algebra: A Constructive Approach

load ex2-4-4 x1 x2 x3 x4 1
T = totbl(A,b); y1 = 1 −1 0 1 −1
y2 = 1 0 1 0 −1
y3 = 1 1 2 −1 0

T =ljx(T,2,1); y2 x2 x3 x4 1
y1 = 1 −1 −1 1 0
x1 = 1 0 −1 0 1
y3 = 1 1 1 −1 1

T =ljx(T,3,2); y2 y3 x3 x4 1
y1 = 2 −1 0 0 1
x1 = 1 0 −1 0 1
x2 = −1 1 −1 1 −1

At this point we are blocked—we cannot pivot y1 to the top of the tableau because the
pivot elements corresponding to x3 and x4 are both zero. (It makes no sense to exchange
y1 with either y2 or y3 since such a move would not increase the number of yi ’s at the top
of the tableau.) Since the element in the final column of the row labeled y1 is nonzero, the
system has no solution because, whenever y2 = 0 and y3 = 0, it follows that y1 = 1. We
are unable to set y1 = y2 = y3 = 0 in the final tableau. In fact, we have

y1 (x) = 2y2 (x) − y3 (x) + 1.

Note that the tableau also shows that the rows of the coefficient matrix A of the original
linear system are related by A1· = 2A2· − A3· and thus are linearly dependent.

Example 2-4-5. We now consider a modification of Example 2-4-4 in which the right-hand
side of the first equation is changed. Our system is now as follows:

x1 − x2 + x4 = 2,
x1 + x3 = 1,
x1 + x2 + 2x3 − x4 = 0.

The data file ex2-4-5.mat can be loaded into MATLAB enabling the following sequence
of tableaus to be constructed:

load ex2-4-5 x1 x2 x3 x4 1
T = totbl(A,b); y1 = 1 −1 0 1 −2
y2 = 1 0 1 0 −1
y3 = 1 1 2 −1 0
2.4. Exact Solution of m Equations in n Unknowns 37

T =ljx(T,2,1); y2 x2 x3 x4 1
y1 = 1 −1 −1 1 −1
x1 = 1 0 −1 0 1
y3 = 1 1 1 −1 1

T =ljx(T,3,2); y 2 y3 x 3 x4 1
y1 = 2 −1 0 0 0
x1 = 1 0 −1 0 1
x2 = −1 1 −1 1 −1

This system has infinitely many solutions because the final column of the tableau
contains a zero in the location corresponding to the yi column label and some of the xj ’s are
still independent variables, and therefore their values can be chosen arbitrarily. Following
the procedure above, we characterize the solution set by allowing x3 and x4 to be arbitrary
and defining
x1 = −x3 + 1,
x2 = −x3 + x4 − 1.
Another way to express this result is to introduce arbitrary variables λ1 and λ2 (representing
the arbitrary values x3 and x4 , respectively) and writing the solution as
       
x1 1 −1 0
x2  −1 −1 1
  =   +   λ1 +   λ 2 .
 x3   0   1  0
x4 0 0 1
Exercise 2-4-6. Using the MATLAB function ljx, solve the following systems of equations
by carrying out all pivot operations. If a system has no solution, give a reason. If a system
has infinitely many solutions, describe the solution set as in Example 2-4-5. If the rows of
the matrix are linearly dependent, write down the actual linear dependence relations.
1. Ax = a, where
   
2 −1 1 1 1
A = −1 2 −1 −2 , a = 1 .
4 1 1 −1 5

2. Bx = b, where    
1 −1 1 2 2
B = 1 1 0 −1 , b = 1 .
1 −3 2 5 1
3. Cx = c, where    
1 −1 1 3
2 1 1 2
C=
−1
, c= 
 2 .
−1 2
1 1 −1 −1
Another Random Scribd Document
with Unrelated Content
 Das Lied der Mutter.
Friedel war ein lieber, lieber Bub, den alle Leute gut leiden
mochten. Immer war er freundlich und lustig wenn er mit seinen
Spielkameraden zusammen war. Schon von weitem hörte man sein
helles Lachen. Nur, wenn er allein im Zimmer saß, oder in der Laube
hinten im Garten, oder am Bach, wo die Fischlein springen, da
wurde er oft ganz still und traurig und mußte weinen. Dann dachte
er an sein Mütterlein, das er so lieb gehabt hatte, so lieb, wie man
gar nicht beschreiben kann, und das nun nicht mehr bei ihm war,
weil der liebe Gott es zu sich in den schönen Himmel genommen
hatte. Solch ein Mütterlein, wie sein Mütterlein gewesen war, hatte
kein anderes Kind das wußte er immer besser, je älter er wurde. So
schön war Mutter gewesen, wie ein Engel. Und so herrliche Lieder
hatte sie gesungen den ganzen Tag, wie die Vögel im Walde es tuen.
Des Abends hatte sie ihn in den Schlaf gesungen mit einem Lied, das
war eigentlich gar kein Lied, es war wie ein süßer Hauch. Es hatte
keine Worte gehabt – nur Töne, aber nie wieder hatte er ein so
schönes Lied gehört. Den Anfang hatte er wohl behalten und
summte ihn oft vor sich hin, wenn er ganz allein war. Aber wenn er
an eine bestimmte Stelle kam, dann wußte er nicht weiter. Dann
quälte er sich und suchte in seinem Kopfe nach den fehlenden
Tönen, aber er fand sie nicht und war sehr traurig darüber. Als er
größer wurde, zeigte es sich, daß er der Mutter Talent für die Musik
geerbt hatte. Mit Leichtigkeit lernte er die Noten, die auf dem Papier
die Töne darstellen, und lernte sie auf dem Klavier zu spielen und sie
zu singen.
Nun suchte er in allen Musikheften, ob er nicht in einem von ihnen
das Lied der Mutter fände. Er suchte, suchte, aber fand es nicht. Da
weinte er oft bitterlich.
 Das sah nun sein Mütterlein im Himmel droben und hatte keine
Ruhe, weil ihr Kind sich so sehr grämte und sie ihm nicht helfen
konnte. Darum bat sie den lieben Herrgott: »Bitte, bitte, laß mich
nur solange wieder hinunter auf die Erde, bis ich dem Friedel mein
Wiegenlied gelehrt habe. Er kann es ja nirgends finden, mein armer
Bub. Es steht ja nirgends geschrieben. Ich habe es mir ja selber für
ihn ausgedacht.«
Als der liebe Gott den Schmerz der Mutter sah, sprach er: »Dein
Sohn soll nicht länger weinen. Schwebe in der Nacht zu ihm hinunter
und singe ihm Dein Liedchen vor.«
Ueberselig war da die Mutter und bedankte sich bei Gott Vater.
Als die Nacht kam, saß die Mutter als schöner, lichter Engel an
Friedels Bett und sang ihm das Lied so lange vor, bis auch er es bis
zu Ende singen konnte. Wie selig war da Friedel! Aber am Morgen,
als er erwachte und sah, daß er ganz allein war und merkte, daß
alles nur ein Traum gewesen, versuchte er wenigstens das Liedchen
zu singen, wie er es im Traume so schön gekonnt hatte. Da mußte er
nun erst recht bitterlich weinen, denn mitten im Gesang blieb er
stecken. Den Schluß hatte er wieder vergessen. Oben im Himmel
aber saß seine Mutter traurig auf einem Wolkenkissen und dachte
darüber nach, wie sie ihren Friedel von seinem Kummer erlösen
könnte.
Da vernahm sie ein leises Rauschen in den Lüften, und als sie aus
dem Himmelsfenster schaute, sah sie, daß ein feiner Regen zur Erde
niederging. »Ihr Regentröpfchen!« sprach sie, »die Ihr die Erde
befruchtet, daß sie Blüten und Früchte trägt, die Ihr den Menschen
so viel Segen bringt, bringt meinem Friedel das Lied seiner Mutter.«
Und sie neigte sich weit aus dem Himmelsfenster hinaus und sang
den Tröpflein ihre Melodie vor.
Die trugen sie hinunter zur Erde und: »Punk – punk! Punk –
punk!« hörte es Friedel in die Dachrinne und gegen die
Fensterscheiben schlagen und dachte: das klingt mir doch so
bekannt – wie ein Lied! –
 Daß es das Lied seines Mütterleins war, das verstand er nicht.
Da verzweifelte die Mutter im Himmel und in ihrer Not bat sie den
Abendwind, sich ihres Friedels anzunehmen, ihm ihr Schlummerlied
vorzusingen. Der lauschte aufmerksam ihrem Gesang und versprach,
sein Bestes zu tun.
In den Aesten der großen Linde vor Friedels Zimmerchen rauschte
und flüsterte er und hatte keinen Ton des schönen Liedes vergessen.
Wohl horchte Friedel auf und dachte: Heute weht der Wind einmal
schön! Aber das Lied der Mutter erkannte er nicht.
Da winkte die Mutter einem Zug Schwalben, der am blauen
Himmel hin und wieder flog. Die Vöglein kamen gehorsam herbei
und ließen sich von dem Leid des schönen Engels erzählen. Dann
steckten sie ein Weilchen die Köpfe zusammen und wippten mit den
Schwänzen und endlich zwitscherten sie: »Wir erfüllen Deine Bitte
ganz gewiß. Wozu nennt man uns denn sonst die Liebesboten?
Wenn Friedel auch unseren Gesang nicht versteht – wir machen uns
ihm auch anders verständlich. Wir kennen die Welt und ihre
Einrichtungen ja so genau. Sei nur ganz ruhig und baue auf uns!«
 Am nächsten Tage lag Friedel im Garten auf dem Rasen und
träumte in die blaue Luft hinein, in der die Schwälblein munter
singend hin und her huschten. Aber er hörte nicht auf ihren Gesang.
Er war in Gedanken wie so oft mit Mütterleins Lied beschäftigt und
summte erst leise, dann immer lauter die ihm bekannten Töne, und
es war, als wenn das Lied eine Leiter wäre, auf der er höher, immer
höher zum Mütterlein steigen könnte. Aber plötzlich konnte er nicht
weiter, das Lied brach mitten drin ab, die Leiter war zu Ende. Vor
Kummer schlug er nach den Blumen im Grase und biß mit den
Zähnen in die Halme. Endlich aber blickte er wieder unverwandt
sehnsüchtig in die Wolken hinauf. Da bemerkte er über sich die
langen Linien der Telephondrähte und wünschte in seinem Herzen,
daß solch ein Draht doch auch in den Himmel hinaufführen möge.
Wie schön wäre das! Dann würde er den ganzen Tag über mit
Mutter telephonieren und den Hörer gewiß erst wieder anhängen,
wenn es Abend wäre und er ins Bett müßte.
Er konnte den Blick nicht von den blanken Drähten abwenden, auf
denen sich eben die Schwälblein niederließen. Nun saßen sie einzeln
und in kleinen Gesellschaften beieinander im Sonnenschein und
rührten sich nicht, und sangen nicht. Ordentlich feierlich still war es
in der Luft. Gebannt hing Friedels Blick an all den schwarzen
Punkten auf den dünnen Linien.
»Ei, das sind ja richtige Noten,« rief er aus und sprang vom Boden
auf. »Das ist ja, das ist ja!« Jetzt sang er selig die Noten nach. Dann
klatschte er in die Hände und jubelte laut: »Das ist ja Mutters Lied!«
Und die Vöglein saßen ganz still, bis Friedel das Lied so oft gesungen
hatte, daß er es ganz genau kannte. Da winkte er ihnen selig zu und
rief: »Habt Dank, habt Dank, Ihr lieben Schwälblein, nun vergesse
ich es niemals, niemals wieder!« Und Mutter droben im Himmel war
nun ganz glücklich, daß ihr Friedel sich nicht mehr zu grämen
brauchte und immer, wenn er sein Liedel so recht von Herzen sang,
daß Blumen, Vögel und Menschen ihre Freude daran hatten, stimmte
Mütterlein glücklich mit ein, und bald sangen alle Englein im Himmel
es mit. Die Schwälblein aber sind zwitschernd von dannen geflogen.
Und eines ist an mein Fenster gekommen und hat mir die ganze
schöne Geschichte erzählt.
 Die Nesselhexe.
Es war im Lande der blauen Sterne. Wohin man blickte, schauten
sie vom Boden auf wie Tausende lieber, blauer Kinderaugen. Als
wäre ein Stück des Himmels zur Erde gesunken, so sahen die
breiten, breiten Felder des Flachses aus, und die Menschen, die ihn
gesät hatten, waren glücklich, denn sie wußten nun, daß sie eine
reiche Ernte haben würden. Sie waren arme Weber und bauten sich
selber den Flachs, aus dem sie die schöne, glatte Leinewand
herstellten. Aber die Freude der armen Leute wurde vernichtet. Krieg
überzog das Land. Wilde, berittene Soldatenhorden rasten über die
Felder, über die blauen Sterne hin und scharfe Pferdehufe zertraten
die Hoffnung der Webersleute. Da ging ein Jammern und Wehklagen
durch die Hütten. Weinend standen Männer und Weiber vor den
zertrampelten Feldern und manch wilder Fluch wurde zum Himmel
geschickt. Die sonst fromm und gläubig gewesen waren, zürnten
dem Herrgott und verlernten das Beten. Nur die Gret, die junge
Witwe, die mit ihren beiden Kindern hinten am Waldesrande in einer
winzigen Hütte wohnte, fluchte und jammerte nicht. Sie weinte nur
heimlich und still vor sich hin, wenn sie an kommende Tage des
Elends und der Not dachte. Auch ging sie fleißig zur Kirche wie
bisher und betete zu Gott, daß er ihr helfe und sie und ihre Kleinen
vor dem Hungertode bewahre. Abends las sie eifrig in dem
Gebetbuche, das sie von ihrer seligen Großmutter geerbt hatte.
Eines Tages, als sie es wieder zur Hand nahm und es aufschlug, fand
sie darin einen vergilbten Zettel, den sie in den langen, langen
Jahren noch niemals gesehen hatte. Auf dem stand von unbekannter
Hand geschrieben: »Bete und arbeite«.
Lange saß die Gret und dachte nach, woher diese Mahnung wohl
kommen mochte, bis es ihr klar wurde, daß Gott sie ihr gesandt
haben mußte. Von diesem Tage an war sie noch fleißiger im Gebet
und in der Arbeit. Es nahte die Zeit der Beerenlese. Da zog sie früh
mit ihren Kindern aus und sammelte mit ihnen am Bergeshang in
glühendem Sonnenbrand lachende Erdbeeren, Heidelbeeren und
Himbeeren, die sie in der nächsten Stadt für schönes Geld verkaufte.
Dafür erwarb sie dann Brot zum Lebensunterhalt.
Aber die Leute im Dorfe wunderten sich, daß sie so zufrieden blieb
und niemals dem Herrgott wegen ihres Unglückes zürnte, und
verhöhnten sie darum.
Als der Sommer zu Ende ging und sie nun wohl zum letzten Male
Beeren suchte, hörte sie hoch oben in den Felsen ein jammervolles
Klagen. Sie suchte und fand ein Ziegenböckchen, das sich verstiegen
hatte und mit gebrochenem Fuß im Grase lag. Behutsam trug sie es
heim, machte ihm im Schuppen neben der Hütte ein weiches Lager
und kühlte liebevoll das schmerzende Glied. Die Nachbarn kamen
und schlugen die Hände zusammen und lachten und riefen: »Was
willst Du denn mit dem Elendswurm? Das wird doch nie wieder
gesund und Milch gibt es doch auch keine, weil's ein Bock ist.«
 Aber die Grete kehrte sich nicht an ihre Reden und sorgte weiter
liebevoll für das Tierchen.
Am Tage trug sie es zum Wiesenrain und ließ es dort grasen und
oft, wenn es in der Nacht vor Schmerzen jammerte, sprang sie aus
dem Bett, um ihm das Bein zu kühlen. Eines Nachts, als das Böcklein
ganz besonders schmerzlich klagte, eilte sie flink zu ihm in den
Schuppen und dachte nicht anders, als daß ihre Augen verzaubert
wären. Golden leuchteten die Hörner des Böckchens aus der Tiefe
des Stalles ihr entgegen, und leuchtend heller Schein lag hinter dem
Tierchen. Als sie zitternd näher trat, öffnete es das rosige
Schnäuzchen und sprach: »Gret, brave Gret! Achte auf das, was ich
nicht fresse, und denke, es wäre Flachs!« Sogleich wurde es dunkel
im Stalle und auch die Hörner des Ziegenbockes leuchteten nicht
mehr golden. Um Mittag trug sie wie immer das Zicklein hinaus zum
Wiesenrain, ließ es äsen, legte sich selber daneben ins Gras und
paßte genau auf. Da bemerkte sie, daß es gierig alle Gräser und
Halme und Blättlein fraß, aber eine Pflanze ließ es stehen, die rührte
es gar nicht an. Das war die Brennessel. Am nächsten Tage geschah
es ebenso und auch am nächstnächsten. Da dachte die Gret: Die
Brennessel ist's, von der ich denken soll, sie wäre Flachs. Wo sie nun
ging und stand, sammelte sie Brennesselpflanzen und trug sie heim
und behandelte sie ganz genau, wie sie es sonst mit dem Flachs
gemacht hatte, und als sie dann die Stengel genau betrachtete, da
sah sie innen darin genau solche graue Fäden wie im Stengel des
Flachses. Da wußte sie, wie reicher Ersatz ihr nun für die verdorbene
Leinernte beschert worden war und sie jubelte vor lauter Glück und
dachte sogleich an ihre vielen, vielen Leidensgenossen, an die
anderen unglücklichen Weber, und zeigte ihnen ihre Entdeckung. Die
aber lachten sie laut schallend aus und verhöhnten sie wieder und
riefen: »Mach nur zu! Das wird ein feines Zeug werden, Dein
Nesselgespinst!«
Die Gret ließ sich jedoch nicht beirren und sammelte fleißig weiter
die verachtete Brennessel. Als der Winter ins Land zog, sahen die
Leute mit Erstaunen die Gret am Fenster sitzen und spinnen. Sie
schüttelten die Köpfe und tuschelten untereinander, die Gret wäre
wohl verrückt geworden – sie wolle Brennesseln spinnen.
Um Weihnachten hörte man aus dem Häuschen der Gret tagaus,
tagein das Klappern des Webstuhles – in allen anderen Hütten stand
er still. Um so lauter gingen die Zungen der Weberleute mit
Schimpfen und Schelten.
Der Winter nahm seinen Abschied mit Sturmgebraus. Heißer
wurden die Strahlen der Sonne und beschienen auf der Wiese vor
Grets Häuschen breite Streifen selbstgewebten Zeuges, die die Gret
dort zum Bleichen hingelegt hatte. Mit ihrer Gieskanne ging sie hin
und her und begoß den Stoff und die Sonne tat ihre Schuldigkeit und
bleichte das Gewebe, bis es weithin leuchtete, weil es so wunderbar
weiß geworden war wie die allerschönste Leinewand. Als das die
Leute im Dorfe sahen, ergriff sie wilder Neid. Wütend drohten sie
der Gret mit den Fäusten über den Zaun hinweg und schrien: »Das
geht nicht mit rechten Dingen zu! Da ist Zauberei dabei!« Und sie
nannten sie von nun an »die Nesselhexe«. Aber die Gret ließ sie
reden und war ihnen nicht böse darum. Im Gegenteil! Sie bot ihnen
freundlich an, ihnen zu zeigen, wie sie es machen mußten, um aus
der verachteten Nessel so schönes Zeug zu gewinnen. Aber sie
hörten nicht auf ihr freundliches Angebot und liefen scheltend von
dannen.
Die Gret war nun im siebenten Himmel mit ihren beiden
Kinderlein. In der Stadt war sie alsbald berühmt für ihre schönen
Nesselstoffe und bekam hohe Preise für dieselben bezahlt, denn
durch das Kriegsunglück konnte man sonst nirgends welche kaufen.
Sie wurde bald eine reiche Frau. Konnte eine Kuh kaufen, die ihr ein
Kälbchen brachte, konnte ein Schweinchen mästen und Hühner
anschaffen und wäre nun ganz glücklich gewesen, hätte sie bei den
anderen Dorfbewohnern nicht so viel Elend und Unglück sehen
müssen. Oft versuchte sie es ganz schüchtern, sie zur Verwertung
der Brennesselfaser zu bewegen, aber immer vergebens, und nicht
selten hörte sie sie hinter sich herrufen: »Nesselhexe! Nesselhexe!«
Was aber alles gute Zureden und alle Liebe der Gret nicht
vermochte, das erreichte endlich die liebe Eitelkeit bei den Leuten.
Als eines Tages, wie jetzt immer, alle Dorfbewohner in schäbigem,
zerfetztem Gewand zur Kirche kamen, erschienen Gret und ihre
kleinen Mädchen in herrlichen neuen, seidenglänzenden Kleidern. Da
reckten die Leute die Hälse und sperrten vor Neid und Staunen die
Mäuler auf und konnten sich nicht sattsehen an den feinen Stoffen.
Als der Gottesdienst vorüber war, schlichen sich die Dorfkinder leise
an Grets Kinder heran, befühlten ihre feinen Kleider. Und auch die
Frauen traten herzu und betrachteten mit großen, glänzenden Augen
den Feststaat der Gret. Anderen Tages erschien in Grets Hütte eine
eitle Frau, um ihr Geheimnis zu erfragen, am nächsten kamen gar
vier und so wurden es immer mehr. Sie sagten, ihre Kinder hätten
ihnen keine Ruhe gelassen, sie wollten auch so schön gekleidet
gehen. Im Grunde war es aber nur die liebe Eitelkeit, die sie dazu
zwang – sie gestanden es nur nicht ein.
So kam es, daß im nächsten Sommer alle Einwohner des Dorfes
Brennesseln sammelten, um daraus schöne Stoffe zu machen – daß
nun alle Not ein Ende hatte.
Die Gret hieß nun aber nicht mehr »die Nesselhexe«. Sie wurde
weit und breit geachtet und geliebt.
Ihr Ziegenböcklein aber, das ihr so viel Glück ins Haus gebracht,
hat sie hoch in Ehren gehalten und liebevoll gepflegt bis zu seinem
Tode.
 Das Guckerohr.
Was man ein Sonntagskind nennt, das wißt ihr Kinder ja schon
lange und auch, daß ein solches sieht, hört und erlebt, was andere
Menschenkinder nicht sehen, hören und erleben.
Heute will ich Euch einmal erzählen, was einem Sonntagskinde an
einem schönen Sommermorgen im Walde begegnete.
Flipsius war schon ein älterer Herr, aber noch gesund und munter,
seines Zeichens Glasschleifer.
Im Winter saß er fleißig hinter dem Tische in seiner Werkstatt und
schliff die herrlichsten Glassachen.
Wenn aber der Frühling ins Land kam, Berge und Täler das schöne
grüne Kleid mit den bunten Blümlein darin anzogen, dann duldete es
Flipsius nicht mehr zu hause.
Dann zog er aus in Gottes Wunderwelt und lauschte und horchte
und äugte, daß ihm ja keine von allen ihren Schönheiten entginge.
Es war im Hochsommer vor Sonnenaufgang, da griff Flipsius zum
Wanderstabe und schlich zum Hause hinaus. Er wollte einmal sehen,
wie es in aller Herrgottsfrühe draußen in der Natur zugeht. Mit ganz
leisen, vorsichtigen Schritten, daß er ja in der Dämmerung kein
Würmchen und kein Blümlein zertrete, betrat er den grünen Wald.
Da war es so still wie in der Kirche. Unter grünen Farnblättern
schliefen Eidechsen und Frösche. Behutsam verkroch er sich unter
die breiten Blätter des Huflattich und wartete geduldig. Endlich
erscholl von der höchsten Spitze einer Tanne der leise Pfiff eines
Vögleins – und wieder – und noch einmal.
 Da reckte Flipsius vorsichtig die Nase aus seinem Versteck hervor,
roch die würzige Morgenluft und sah allzugleich, daß Frau Sonne mit
ihren ersten, rosigen Strahlen die Kronen der Bäume streichelte.
Wups verkroch er sich wieder hinter die Blätter. Er wollte ja sehen,
aber selber nicht gesehen werden.
Jetzt ließen schon mehrere Vöglein ihre Stimme erschallen und als
gar ein Nußhäher mit lautem Schrei sein Erwachen ankündigte, da
wurde es rings lebendig.
Dicht neben Flipsius regte es sich. Eine Amsel kam geschäftig
dahergetrippelt und machte sich mit Klopfen und Pochen an der
breiten Spalte in der Rinde einer morschen Eiche zu schaffen bis aus
dem Innern des Stammes ein feines Stimmchen antwortete.
Da bog Frau Amsel mit ihrem Schnabel ein Farnkraut beiseite, das
den Eingang versperrte und heraus schlüpfte ein Elflein zart und
rosig, rieb sich die Augen, gähnte recht herzhaft und schaute umher,
ob auch kein Lauscher in der Nähe sei.
 Dann hüpfte es vergnügt zu einem großen Fliegenpilz, in dessen
Mitte sich eine Menge Tautropfen gesammelt hatten, ließ das
Hemdlein aus allerfeinstem Nebelbatist herunter bis auf die zarten
Füße und wusch Gesicht, Brust und Arme, daß es eine Lust war.
Flipsius lächelte und hielt den Atem an – es sah ja gar zu lieblich
aus.
Als nun das ganze Geschöpfchen schön sauber war bis hinter die
winzigen Ohren, griff die kleine Elfe zum Handtuch aus
Spinnengewebe, das nebenan auf einer Distelstaude hing und
trocknete und rieb sich ab; zuletzt kamen die Aeuglein daran. Da
stieß sie einen gellenden Schrei aus, so daß Flipsius sich vor Schreck
beinahe verraten hätte, hielt beide Hände vor's Gesicht und rief:
»Mein Auge! Oh mein Auge! Helft mir doch! Herbei! Herbei!«
Da kamen von allen Seiten viele, viele andere Elflein herbeigeeilt,
besahen das kranke Auge, betrachteten das Handtuch, das an dem
Schmerz schuld sein sollte und rangen ratlos die Hände.
Endlich erschien Frau Amsel, brachte auf einem Frauenmantelblatt
kühlenden Morgentau für das geschwollene Auge und sagte: »Ich
hole den Doktor Humpelpumpel. Mit Eurem Heulen und Zetern ist da
nicht geholfen!« und flog eilends davon.
Nun konnten die kleinen Elflein sich nicht genug tun mit Bedauern
und Streicheln und »ach! und oh!«
Die arme Kranke saß mitten unter ihnen auf einem Baumstumpf
und weinte bitterlich. Als dann endlich der Doktor Humpelpumpel
kam, wußte Flipsius sogleich, daß er zum Geschlecht der Zwerge
gehörte und warum er den Namen Humpelpumpel führte. Er hatte
einen kurzen Fuß und humpelte ganz fürchterlich. Dafür aber hatte
er eine riesengroße, spitze Nase und sah dadurch sehr gescheit aus.
Er besah sich sofort das kranke Auge, runzelte die Stirne, pfiff durch
die Zähne und lachte.
»Da haben wir ja schon den Schaden! Einen Augenblick!« Mit
spitzen Fingern griff er zu. Elflein schrie herzzerreißend.
»Da, da ist das ganze Malheur – ein Spinnenbein war im Handtuch
gehängt und hat sich im Augapfel festgehakt!« sagte er. »Ich habe
es natürlich sofort entdeckt!« Stolz warf sich Dr. Humpelpumpel in
die Brust. »Morgen früh komme ich wieder nachsehen. Bis dahin
kühlen Sie das Auge nur weiter, mein schönes Fräulein!« Hier
machte er eine herrliche Verbeugung und verschwand.
»Ein Spinnenbein!« riefen die Elfenfreundinnen. »Wer hätte das
gedacht? Ja, ja, man kann nicht vorsichtig genug sein!«
 Nun wartete Flipsius bis das ganze kleine Volk sich in den Schatten
verkrochen hatte, dann ging er vergnügt heim. Was er erlebt hatte,
das sah doch nur ein Sonntagskind. Wie gut ist's, am Sonntag
geboren zu sein!
Als er am nächsten Morgen wieder ganz leise in sein Versteck
kroch und durch die Blätter äugte, saß Elflein schon auf seinem
Baumstumpf mit einem großen Verband über dem Auge und wartete
auf seinen Leibarzt. Hin und wieder stöhnte es leise und seufzte ein
wenig. Als endlich der Herr Doktor erschien im schönen, neuen,
hellgrauen Sommeranzug, da ließ Elflein gar das Köpflein tief auf die
Brust sinken und tat als wenn's mit ihm zu Ende ginge. Der Herr
Doktor lächelte und streichelte ein ganz klein wenig das Händchen
der Patientin. Dann entfernte er den Umschlag.
»Ja,« rief er, daß es laut im Walde widerhallte, »ja, was wollen Sie
denn, teures Fräulein? Das Auge ist durch meine Pflege ja
vollständig gesund. Da ist keine Spur von einer Erkrankung mehr zu
sehen. Es ist genau so lieblich und schön himmelblau wie das andere
Auge auch!«
Da plinkte und blinzelte Elflein ein wenig und sagte: »Wenn ich
doch aber lange nicht mehr so gut sehe wie vorher! Das ist doch
schrecklich!«
»Nun, eine kleine Sehschwäche bleibt nach solchen Verletzungen
ja gerne zurück. Aber, schönes Fräulein, wozu haben Sie denn den
berühmten Geheimen Sanitätsrat Professor Doktor Humpelpumpel
zum Arzt? Ich weiß auch hier Rat. Sehen Sie einmal her. Sehen Sie
einmal auf dieses Blatt am Boden – hier – und nun betrachten Sie
das gleiche Blatt hier an der Stelle wo der Wassertropfen liegt.
Sehen Sie es da nicht viel deutlicher?«
»Freilich,« meinte Elflein. »Das ist mal nett. Das hatte ich noch nie
bemerkt. Aber was soll mir das helfen? Ich kann doch nicht die
ganze Welt durch dies Tröpflein betrachten!«
»Doch, doch, das ist's ja eben, meine Gnädige. Die Wissenschaft
macht sich dies Tröpflein zu Nutze. Das heißt: Die Wissenschaft bin
in diesem Falle ich. Mit Hülfe solches Tröpfleins stelle ich einen
Sehapparat her – ein Guckerohr. Passen Sie auf.«
Er nahm vom Boden ein dünnes hohles Aestlein, tat einen
Tautropfen hinein, ließ ihn nach vorne an die Spitze der Röhre
gleiten, sah hindurch und rief: »Famos, famos! Schauen Sie einmal
hindurch!«
Das tat Elflein nun und war begeistert.
»Doktor, Doktor, Sie sind ein Schatz! Wenn ich Sie nicht hätte!«
jauchzte sie und gab ihm einen herzhaften Kuß.
Humpelpumpel stand erst wie versteinert. Dann reckte er sich
stolz über Lebensgröße. Einen Elfenkuß hatte er doch noch nie
bekommen. Als der Doktor gegangen war, lief Elflein mit seinem
Guckerohr hin und her. Sah in den blauen Himmel, in die Berge
hinein und wollte nun auch einen Blick ins Tal werfen. Dazu erklomm
es einen kleinen runden Hügel, der ganz und gar mit einem grünen
Moosteppich überzogen war. Sie erklomm ihn und trippelte munter
darauf herum. Hätte sie geahnt, woraus der Hügel bestand, sie hätte
es nicht getan. Sie hätte großes Unglück verhütet.
Solange es eben ging, hielt Flipsius sich tapfer. Als aber die kleinen
zierlichen Füße der Elfe gar zu lebhafte Sprünge machten, hielt er es
nicht länger aus und mußte laut lachen. Er war eben wie die meisten
Menschen kitzelig und der kleine, runde Hügel, auf dem Elflein
stand, war sein Bauch, der in einer grünen Sammetweste steckte.
Kaum hatte er das schallende Gelächter ausgestoßen, da stürzte
Elflein vor Schreck von seinem schönen Aussichtshügel herunter und
als es davoneilen wollte, schrie es laut auf. Es konnte ja nicht
aufstehen. Es hatte sich bei dem Sturze den Fuß gebrochen. Zum
Glück hatte Dr. Humpelpumpel den Schrei noch vernommen und
kehrte eilig zurück. Als er aber den großen bärtigen Mann im Grase
sitzen sah, erschrak er heftig und wollte davoneilen, aber die Kleine
machte so flehende Augen, daß er sich überwand, zu ihr eilte und
sie auf seinen Armen in das Dickicht trug.
 Da saß nun Flipsius, das Sonntagskind und grämte sich über das
Unheil, das er angerichtet hatte.
»Schade! Schade!« murmelte er und ging betrübt nach Hause.
Dort überlegte er noch einmal das ganze Erlebnis und als er an das
Guckerohr dachte, das Humpelpumpel aus einem Aestchen und
einem Tautropfen gemacht hatte, kam ihm ein guter Gedanke. Er
konnte ja so viele schöne Dinge aus Glas machen – warum sollte er
nicht auch einen schönen, klaren, großen Tropfen schleifen können?
Sogleich ging er an die Arbeit und siehe – in wenigen Wochen war
das Kunstwerk gelungen. Den Glastropfen faßte er in eine blanke
Metallröhre und Flipsius wurde nun weit und breit als Erfinder des
Fernrohres geehrt und bewundert. Von Dr. Humpelpumpel, der doch
eigentlich zuerst auf den Gedanken gekommen war, wußte niemand.
So ergeht es eben den meisten Erfindern auf dieser Welt.
Wollt Ihr Euch solch ein gläsernes Tröpflein einmal genau
betrachten, so seht nur das Theaterglas an, das Mutter im Beutel
hat, wenn sie schön geputzt in die Oper geht – oder Vaters Brille, die
hat zwei solche geschliffene Glastropfen.
 Berggeistchens Talfahrt.
Auf den höchsten Höhen des Gebirges, dort, wo ewiger Schnee
und ewiges Eis leuchten, herrscht der Berggeist. Kein menschliches
Auge hat ihn je gesehen, aber die Bäche, die von dort oben mit
Donnergepolter heruntertosen, die erzählen, daß er einen langen,
silberweißen Bart hat und sein Gewand so grau ist wie das älteste
Gestein. Auf dem Haupte trägt er eine Krone von leuchtenden
Bergkrystallen. Er regiert dort oben über Wetter und Winde, über
Blitz und Donner. Und damit er seinen Willen weithin über alle Gipfel
kund tuen kann, hat er unter sich viele, viele kleine, drollige
Berggeistchen, die müssen für ihn laufen und springen. Müssen das
Echo wecken, wenn es eingeschlafen ist und die breiten,
schimmernden Nebeltücher hin und her ziehen, vor die Sonne und
wieder zurück, ganz wie der alte Berggeist befiehlt.
Sie haben einen strengen Dienst, die Kleinen, und wenig Zeit für
ihre eigenen Späßchen und Vergnügungen. Am meisten freut es sie,
wenn sie Schneebälle den Berg hinuntertrudeln lassen dürfen. Die
machen dabei so lustige Hopser und Sprünge und werden im
Hinabrollen größer und größer, bis sie so groß sind, wie ein Haus.
Dann stürzen sie mit Donnergepolter ins Tal hinab und nehmen
dabei Felsen und Bäume mit. Die Menschen nennen einen solchen
Riesenschneeball Lawine und haben große Angst davor. Nun war
einmal unter der Schar der Berggeistchen ein besonders pfiffiger und
lustiger. Dem wurde es bei der vielen Arbeit und dem ewigen Hin-
und Herlaufen zu langweilig. Besonders seit er einmal ein
Menschenpaar gesehen hatte, das mit Eispickel und Stöcken
bewaffnet, jubelnd und jauchzend auf der allerhöchsten Bergspitze
gestanden war. Da packte ihn die Sehnsucht, mehr von der Welt zu
sehen als nur die öden Gebirgshöhen. Er wollte in das Tal hinab,
Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about books and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!

ebookgate.com