Python Algorithms Mastering Basic Algorithms In The Python Language Second Edition 2nd Edition Magnus Lie Hetland download
Python Algorithms Mastering Basic Algorithms In The Python Language Second Edition 2nd Edition Magnus Lie Hetland download
https://ebookbell.com/product/python-algorithms-mastering-basic-
algorithms-in-the-python-language-second-edition-2nd-edition-
magnus-lie-hetland-50200310
https://ebookbell.com/product/python-algorithms-mastering-basic-
algorithms-in-the-python-language-second-edition-magnus-lie-
hetland-10817504
https://ebookbell.com/product/python-algorithms-mastering-basic-
algorithms-in-the-python-language-second-edition-2nd-magnus-lie-
hetland-7384184
https://ebookbell.com/product/python-algorithms-mastering-basic-
algorithms-in-the-python-language-hetland-21889438
https://ebookbell.com/product/python-algorithms-mastering-basic-
algorithms-in-the-python-language-2nd-edition-hetland-38431100
300 Python Algorithms Mastering The Art Of Problemsolving Hernando
Abella
https://ebookbell.com/product/300-python-algorithms-mastering-the-art-
of-problemsolving-hernando-abella-63693184
https://ebookbell.com/product/300-python-algorithms-mastering-the-art-
of-problemsolving-hernando-abella-79087836
https://ebookbell.com/product/300-python-algorithms-mastering-the-art-
of-problemsolving-hernando-abella-80392536
https://ebookbell.com/product/mastering-opencv-with-python-use-numpy-
scikit-tensorflow-and-matplotlib-to-learn-advanced-algorithms-for-
machine-learning-through-a-set-of-practical-projects-ayush-
vaishya-53747356
https://ebookbell.com/product/coding-black-scholes-mastering-
algorithmic-options-trading-with-python-van-der-post-56240580
Python Algorithms
Second Edition
Trademarked names, logos, and images may appear in this book. Rather than use a trademark
symbol with every occurrence of a trademarked name, logo, or image we use the names,
logos, and images only in an editorial fashion and to the benefit of the trademark owner, with
no intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even
if they are not identified as such, is not to be taken as an expression of opinion as to whether
or not they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date
of publication, neither the authors nor the editors nor the publisher can accept any legal
responsibility for any errors or omissions that may be made. The publisher makes no
warranty, express or implied, with respect to the material contained herein.
Publisher: Heinz Weinheimer
Lead Editor: Steve Anglin
Technical Reviewer: Stefan Turalski Editorial Board: Steve Anglin, Mark
Beckner, Ewan Buckingham, Gary Cornell, Louise Corrigan, James T.
DeWolf, Jonathan Gennick, Robert Hutchinson, Michelle Lowman, James
Markham, Matthew Moodie, Jeff Olson, Jeffrey Pepper, Douglas Pundick, Ben
Renow-Clarke, Dominic Shakeshaft, Gwenan Spearing, Matt Wade, Steve
Weiss Development Editor: Kenyon Brown Coordinating Editor: Anamika
Panchoo Copy Editor: Kim Wimpsett
Compositor: SPi Global
Indexer: SPi Global
Artist: SPi Global
Cover Designer: Anna Ishchenko Photo Credit: Kai T. Dragland
Distributed to the book trade worldwide by Springer Science+Business Media New York,
233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-
4505, e-mail orders-ny@springer-sbm.com, or visit
www.springeronline.com. Apress Media, LLC is a California LLC and the sole
member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc).
SSBM Finance Inc is a Delaware corporation.
Apress and friends of ED books may be purchased in bulk for academic, corporate, or
promotional use. eBook versions and licenses are also available for most titles. For more
information, reference our Special Bulk Sales–eBook Licensing web page at
www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this text is
available to readers at www.apress.com. For detailed information about how to locate
your book’s source code, go to www.apress.com/source-code/.
For JV, the silliest girl I know.
Contents at a Glance
Preface
Chapter 1: Introduction
Chapter 2: The Basics
Chapter 3: Counting 101
Chapter 4: Induction and Recursion ... and
Reduction
Chapter 5: Traversal: The Skeleton Key of
Algorithmics
Chapter 6: Divide, Combine, and Conquer
Chapter 7: Greed Is Good? Prove It!
Chapter 8: Tangled Dependencies and
Memoization
Chapter 9: From A to B with Edsger and
Friends
Chapter 10: Matchings, Cuts, and Flows
Chapter 11: Hard Problems and (Limited)
Sloppiness
Appendix A: Pedal to the Metal: Accelerating
Python
Appendix B: List of Problems and Algorithms
Appendix C: Graph Terminology
Appendix D: Hints for Exercises
Index
Contents
Preface
Chapter 1: Introduction
What’s All This, Then?
What the book is about:
What the book covers only briefly or partially:
What the book isn’t about:
Summary
If You’re Curious …
Exercises
References
Chapter 3: Counting 101
The Skinny on Sums
More Greek
Working with Sums
Stronger Assumptions
Invariants and Correctness
Relaxation and Gradual Improvement
Reduction + Contraposition = Hardness Proof
Problem Solving Advice
Summary
If You’re Curious …
Exercises
References
Chapter 5: Traversal: The Skeleton Key of
Algorithmics
A Walk in the Park
No Cycles Allowed
How to Stop Walking in Circles
Go Deep!
Depth-First Timestamps and Topological Sorting (Again)
Sorting by Halves
How Fast Can We Sort?
Three More Examples
Closest Pair
Convex Hull
Greatest Slice
Huffman’s Algorithm
The Algorithm
The First Greedy Choice
Going the Rest of the Way
Optimal Merging
Summary
If You’re Curious …
Exercises
References
Chapter 8: Tangled Dependencies and
Memoization
Don’t Repeat Yourself
Shortest Paths in Directed Acyclic Graphs
Longest Increasing Subsequence
Sequence Comparison
The Knapsack Strikes Back
Binary Sequence Partitioning
Summary
If You’re Curious …
Exercises
References
Chapter 9: From A to B with Edsger and
Friends
Propagating Knowledge
Relaxing like Crazy
Finding the Hidden DAG
All Against All
Far-Fetched Subproblems
Meeting in the Middle
Knowing Where You’re Going
Summary
If You’re Curious …
Exercises
References
Chapter 10: Matchings, Cuts, and Flows
Bipartite Matching
Disjoint Paths
Maximum Flow
Minimum Cut
Cheapest Flow and the Assignment Problem*
Some Applications
Summary
If You’re Curious …
Exercises
References
Chapter 11: Hard Problems and (Limited)
Sloppiness
Reduction Redux
Not in Kansas Anymore?
Meanwhile, Back in Kansas …
But Where Do You Start? And Where Do You Go
from There?
A Ménagerie of Monsters
Return of the Knapsack
Cliques and Colorings
Paths and Circuits
Index
About the Author
Introduction
1. Write down the problem.
2. Think real hard.
Consider the following problem: You are to visit all the cities, towns,
and villages of, say, Sweden and then return to your starting point. This
might take a while (there are 24,978 locations to visit, after all), so you
want to minimize your route. You plan on visiting each location exactly
once, following the shortest route possible. As a programmer, you
certainly don’t want to plot the route by hand. Rather, you try to write
some code that will plan your trip for you. For some reason, however,
you can’t seem to get it right. A straightforward program works well for
a smaller number of towns and cities but seems to run forever on the
actual problem, and improving the program turns out to be surprisingly
hard. How come?
Actually, in 2004, a team of five researchers1 found such a tour of
Sweden, after a number of other research teams had tried and failed.
The five-man team used cutting-edge software with lots of clever
optimizations and tricks of the trade, running on a cluster of 96 Xeon
2.6GHz workstations. Their software ran from March 2003 until May
2004, before it finally printed out the optimal solution. Taking various
interruptions into account, the team estimated that the total CPU time
spent was about 85 years!
Consider a similar problem: You want to get from Kashgar, in the
westernmost region of China, to Ningbo, on the east coast, following
the shortest route possible.2 Now, China has 3,583,715 km of roadways
and 77,834 km of railways, with millions of intersections to consider
and a virtually unfathomable number of possible routes to follow. It
might seem that this problem is related to the previous one, yet this
shortest path problem is one solved routinely, with no appreciable
delay, by GPS software and online map services. If you give those two
cities to your favorite map service, you should get the shortest route in
mere moments. What’s going on here?
You will learn more about both of these problems later in the book;
the first one is called the traveling salesman (or salesrep) problem and
is covered in Chapter 11, while so-called shortest path problems are
primarily dealt with in Chapter 9. I also hope you will gain a rather deep
insight into why one problem seems like such a hard nut to crack while
the other admits several well-known, efficient solutions. More
importantly, you will learn something about how to deal with
algorithmic and computational problems in general, either solving them
efficiently, using one of the several techniques and algorithms you
encounter in this book, or showing that they are too hard and that
approximate solutions may be all you can hope for. This chapter briefly
describes what the book is about—what you can expect and what is
expected of you. It also outlines the specific contents of the various
chapters to come in case you want to skip around.
What’s All This, Then?
This is a book about algorithmic problem solving for Python
programmers. Just like books on, say, object-oriented patterns, the
problems it deals with are of a general nature—as are the solutions. For
an algorist, there is more to the job than simply implementing or
executing an existing algorithm, however. You are expected to come up
with new algorithms—new general solutions to hitherto unseen, general
problems. In this book, you are going to learn principles for
constructing such solutions.
This is not your typical algorithm book, though. Most of the
authoritative books on the subject (such as Knuth’s classics or the
industry-standard textbook by Cormen et al.) have a heavy formal and
theoretical slant, even though some of them (such as the one by
Kleinberg and Tardos) lean more in the direction of readability. Instead
of trying to replace any of these excellent books, I’d like to supplement
them. Building on my experience from teaching algorithms, I try to
explain as clearly as possible how the algorithms work and what
common principles underlie many of them. For a programmer, these
explanations are probably enough. Chances are you’ll be able to
understand why the algorithms are correct and how to adapt them to
new problems you may come to face. If, however, you need the full
depth of the more formalistic and encyclopedic textbooks, I hope the
foundation you get in this book will help you understand the theorems
and proofs you encounter there.
Unless you’ve encountered this situation before, the new code might
look promising, but try to run it. Chances are you’ll notice a distinct
slowdown. On my computer, the second piece of code takes around 200
times as long as the first to finish.5 Not only is it slower, but it also
scales worse with the problem size. Try, for example, to increase
count from 10**5 to 10**6. As expected, this increases the running
time for the first piece of code by a factor of about ten … but the second
version is slowed by roughly two orders of magnitude, making it more
than two thousand times slower than the first! As you can probably
guess, the discrepancy between the two versions only increases as the
problem gets bigger, making the choice between them ever more
crucial.
Some Prerequisites
This book is intended for two groups of people: Python programmers,
who want to beef up their algorithmics, and students taking algorithm
courses, who want a supplement to their plain-vanilla algorithms
textbook. Even if you belong to the latter group, I’m assuming you have
a familiarity with programming in general and with Python in
particular. If you don’t, perhaps my book Beginning Python can help?
The Python web site also has a lot of useful material, and Python is a
really easy language to learn. There is some math in the pages ahead,
but you don’t have to be a math prodigy to follow the text. You’ll be
dealing with some simple sums and nifty concepts such as polynomials,
exponentials, and logarithms, but I’ll explain it all as we go along.
Before heading off into the mysterious and wondrous lands of
computer science, you should have your equipment ready. As a Python
programmer, I assume you have your own favorite text/code editor or
integrated development environment—I’m not going to interfere with
that. When it comes to Python versions, the book is written to be
reasonably version-independent, meaning that most of the code should
work with both the Python 2 and 3 series. Where backward-
incompatible Python 3 features are used, there will be explanations on
how to implement the algorithm in Python 2 as well. (And if, for some
reason, you’re still stuck with, say, the Python 1.5 series, most of the
code should still work, with a tweak here and there.)
If You’re Curious …
This is a section you’ll see in all the chapters to come. It’s intended to
give you some hints about details, wrinkles, or advanced topics that
have been omitted or glossed over in the main text and to point you in
the direction of further information. For now, I’ll just refer you to the
“References” section, later in this chapter, which gives you details about
the algorithm books mentioned in the main text.
Exercises
As with the previous section, this is one you’ll encounter again and
again. Hints for solving the exercises can be found in Appendix D. The
exercises often tie in with the main text, covering points that aren’t
explicitly discussed there but that may be of interest or that deserve
some contemplation. If you want to really sharpen your algorithm
design skills, you might also want to check out some of the myriad of
sources of programming puzzles out there. There are, for example, lots
of programming contests (a web search should turn up plenty), many of
which post problems that you can play with. Many big software
companies also have qualification tests based on problems such as these
and publish some of them online.
Because the introduction doesn’t cover that much ground, I’ll just
give you a couple of exercises here—a taste of what’s to come:
References
Applegate, D., Bixby, R., Chvátal, V., Cook, W., and Helsgaun, K.
Optimal tour of Sweden.
www.math.uwaterloo.ca/tsp/sweden/. Accessed April
6, 2014.
The Basics
Asymptotic Notation
Remember the append versus insert example in Chapter 1?
Somehow, adding items to the end of a list scaled better with the list
size than inserting them at the front; see the nearby “Black Box” sidebar
on list for an explanation. These built-in operations are both written
in C, but assume for a minute that you reimplement list.append in
pure Python; let’s say arbitrarily that the new version is 50 times slower
than the original. Let’s also say you run your slow, pure-Python
append-based version on a really slow machine, while the fast,
optimized, insert-based version is run on a computer that is 1,000 times
faster. Now the speed advantage of the insert version is a factor of
50,000. You compare the two implementations by inserting 100,000
numbers. What do you think happens?
Intuitively, it might seem obvious that the speedy solution should
win, but its “speediness” is just a constant factor, and its running time
grows faster than the “slower” one. For the example at hand, the
Python-coded version running on the slower machine will, actually,
finish in half the time of the other one. Let’s increase the problem size a
bit, to 10 million numbers, for example. Now the Python version on the
slow machine will be 2,000 times faster than the C version on the fast
machine. That’s like the difference between running for about a minute
and running almost a day and a half!
This distinction between constant factors (related to such things as
general programming language performance and hardware speed, for
example) and the growth of the running time, as problem sizes increase,
is of vital importance in the study of algorithms. Our focus is on the big
picture—the implementation-independent properties of a given way of
solving a problem. We want to get rid of distracting details and get
down to the core differences, but in order to do so, we need some
formalism.
f(n) ≤ cg(n)
for all n ≥ n0. In other words, if we’re allowed to tweak the constant c
(for example, by running the algorithms on machines of different
speeds), the function g will eventually (that is, at n0) grow bigger than f.
See Figure 2-1 for an example.
2 2
Figure 2-1. For values of n greater than n0, T(n) is less than cn , so T(n) is O(n )
2 2 2
in the set O(n2), or, in set notation, n2 ∈ O(n2). We often simply say that
n2 is O(n2).
The fact that n2 does not grow faster than itself is not particularly
interesting. More useful, perhaps, is the fact that neither 2.4n2 + 7 nor
the linear function n does. That is, we have both
2.4n2 + 7 ∈ O(n2)
and
n ∈ O(n2).
for all n ≥ n0. So, where O forms a so-called asymptotic upper bound, Ω
forms an asymptotic lower bound.
Note Our first two asymptotic operators, O and Ω, are each
other’s inverses: If f is O(g), then g is Ω(f). Exercise 2-3 asks you
to show this.
The sets formed by Θ are simply intersections of the other two, that
is, Θ(g) = O(g) ∩ Ω(g). In other words, a function f is in Θ(g) if it
satisfies the following condition: There exists a natural number n0 and
two positive constants c1 and c2 such that
c1g(n) ≤ f(n) ≤ c2g(n)
for all n ≥ n0. This means that f and g have the same asymptotic growth.
For example, 3n2 + 2 is Θ(n2), but we could just as well write that n2 is
Θ(3n2 + 2). By supplying an upper bound and a lower bound at the
same time, the Θ operator is the most informative of the three, and I will
use it when possible.
ebookbell.com