Problem Solving With Algorithms And Data

Structures Using Python Second Edition 2nd

Bradley N Miller download

https://ebookbell.com/product/problem-solving-with-algorithms-
and-data-structures-using-python-second-edition-2nd-bradley-n-
miller-4744554

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Programming For Problemsolving With C Formulating Algorithms For

Complex Problems Kamaldeep

https://ebookbell.com/product/programming-for-problemsolving-with-c-
formulating-algorithms-for-complex-problems-kamaldeep-55111172

Solving Optimization Problems With The Heuristic Kalman Algorithm New

Stochastic Methods Rosario Toscano

https://ebookbell.com/product/solving-optimization-problems-with-the-
heuristic-kalman-algorithm-new-stochastic-methods-rosario-
toscano-56272002

Problem Solving With C Ninth Edition Mock Kenrick Savitch

https://ebookbell.com/product/problem-solving-with-c-ninth-edition-
mock-kenrick-savitch-21355058

Problem Solving With C 10th Edition Savitch Walter J

https://ebookbell.com/product/problem-solving-with-c-10th-edition-
savitch-walter-j-22124074
Problem Solving With C 7th Edition 7th Walter Savitch

https://ebookbell.com/product/problem-solving-with-c-7th-edition-7th-
walter-savitch-2433248

Problem Solving With C 6th Edition 6th Edition Savitch Walter

https://ebookbell.com/product/problem-solving-with-c-6th-edition-6th-
edition-savitch-walter-38476560

Problem Solving With C The Object Of Programming Fourth Edition Walter

J Savitch

https://ebookbell.com/product/problem-solving-with-c-the-object-of-
programming-fourth-edition-walter-j-savitch-7378714

Problem Solving With The Private Sector A Public Solutions Handbook

Daniel E Bromberg

https://ebookbell.com/product/problem-solving-with-the-private-sector-
a-public-solutions-handbook-daniel-e-bromberg-43932284

Problem Solving With C 7th Edition Walter Savitch

https://ebookbell.com/product/problem-solving-with-c-7th-edition-
walter-savitch-43272370
Problem Solving with Algorithms and
Data Structures
Release 3.0

Brad Miller, David Ranum

September 22, 2013

 CONTENTS

1 Introduction 3
1.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 What Is Computer Science? . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Review of Basic Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.6 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.7 Programming Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2 Algorithm Analysis 41
2.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.2 What Is Algorithm Analysis? . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.3 Performance of Python Data Structures . . . . . . . . . . . . . . . . . . . . . 52
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6 Discussion Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.7 Programming Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3 Basic Data Structures 61

3.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2 What Are Linear Structures? . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.3 Stacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.4 The Stack Abstract Data Type . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.5 Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.6 Deques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.7 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.8 The Unordered List Abstract Data Type . . . . . . . . . . . . . . . . . . . . . 98
3.9 Implementing an Unordered List: Linked Lists . . . . . . . . . . . . . . . . . 98
3.10 The Ordered List Abstract Data Type . . . . . . . . . . . . . . . . . . . . . . 108
3.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.12 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.13 Discussion Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.14 Programming Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

4 Recursion 117
4.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.2 What is Recursion? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

i
 4.3 Stack Frames: Implementing Recursion . . . . . . . . . . . . . . . . . . . . . 123
4.4 Visualising Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.5 Complex Recursive Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.6 Exploring a Maze . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.8 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.9 Discussion Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.10 Programming Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

5 Sorting and Searching 147

5.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.2 Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.3 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
5.5 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
5.6 Discussion Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
5.7 Programming Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

6 Trees and Tree Algorithms 185

6.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.2 Examples Of Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.3 Vocabulary and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
6.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
6.5 Priority Queues with Binary Heaps . . . . . . . . . . . . . . . . . . . . . . . 198
6.6 Binary Tree Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
6.7 Tree Traversals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
6.8 Binary Search Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
6.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
6.10 Key Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
6.11 Discussion Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
6.12 Programming Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

7 JSON 235
7.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
7.2 What is JSON? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
7.3 The JSON Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

ii
 Problem Solving with Algorithms and Data Structures, Release 3.0

CONTENTS 1
Problem Solving with Algorithms and Data Structures, Release 3.0

2 CONTENTS
 CHAPTER

ONE

INTRODUCTION

1.1 Objectives

• To review the ideas of computer science, programming, and problem-solving.

• To understand abstraction and the role it plays in the problem-solving process.
• To understand and implement the notion of an abstract data type.
• To review the Python programming language.

1.2 Getting Started

The way we think about programming has undergone many changes in the years since the first
electronic computers required patch cables and switches to convey instructions from human
to machine. As is the case with many aspects of society, changes in computing technology
provide computer scientists with a growing number of tools and platforms on which to practice
their craft. Advances such as faster processors, high-speed networks, and large memory ca-
pacities have created a spiral of complexity through which computer scientists must navigate.
Throughout all of this rapid evolution, a number of basic principles have remained constant.
The science of computing is concerned with using computers to solve problems.
You have no doubt spent considerable time learning the basics of problem-solving and hope-
fully feel confident in your ability to take a problem statement and develop a solution. You have
also learned that writing computer programs is often hard. The complexity of large problems
and the corresponding complexity of the solutions can tend to overshadow the fundamental
ideas related to the problem-solving process.
This chapter emphasizes two important areas for the rest of the text. First, it reviews the frame-
work within which computer science and the study of algorithms and data structures must fit,
in particular, the reasons why we need to study these topics and how understanding these top-
ics helps us to become better problem solvers. Second, we review the Python programming
language. Although we cannot provide a detailed, exhaustive reference, we will give examples
and explanations for the basic constructs and ideas that will occur throughout the remaining
chapters.

3
Problem Solving with Algorithms and Data Structures, Release 3.0

1.3 What Is Computer Science?

Computer science is often difficult to define. This is probably due to the unfortunate use of
the word “computer” in the name. As you are perhaps aware, computer science is not simply
the study of computers. Although computers play an important supporting role as a tool in the
discipline, they are just that – tools.
Computer science is the study of problems, problem-solving, and the solutions that come out
of the problem-solving process. Given a problem, a computer scientist’s goal is to develop an
algorithm, a step-by-step list of instructions for solving any instance of the problem that might
arise. Algorithms are finite processes that if followed will solve the problem. Algorithms are
solutions.
Computer science can be thought of as the study of algorithms. However, we must be careful to
include the fact that some problems may not have a solution. Although proving this statement
is beyond the scope of this text, the fact that some problems cannot be solved is important for
those who study computer science. We can fully define computer science, then, by including
both types of problems and stating that computer science is the study of solutions to problems
as well as the study of problems with no solutions.
It is also very common to include the word computable when describing problems and solu-
tions. We say that a problem is computable if an algorithm exists for solving it. An alternative
definition for computer science, then, is to say that computer science is the study of problems
that are and that are not computable, the study of the existence and the nonexistence of algo-
rithms. In any case, you will note that the word “computer” did not come up at all. Solutions
are considered independent from the machine.
Computer science, as it pertains to the problem-solving process itself, is also the study of
abstraction. Abstraction allows us to view the problem and solution in such a way as to
separate the so-called logical and physical perspectives. The basic idea is familiar to us in a
common example.
Consider the automobile that you may have driven to school or work today. As a driver, a user
of the car, you have certain interactions that take place in order to utilize the car for its intended
purpose. You get in, insert the key, start the car, shift, brake, accelerate, and steer in order to
drive. From an abstraction point of view, we can say that you are seeing the logical perspective
of the automobile. You are using the functions provided by the car designers for the purpose of
transporting you from one location to another. These functions are sometimes also referred to
as the interface.
On the other hand, the mechanic who must repair your automobile takes a very different point
of view. She not only knows how to drive but must know all of the details necessary to carry
out all the functions that we take for granted. She needs to understand how the engine works,
how the transmission shifts gears, how temperature is controlled, and so on. This is known as
the physical perspective, the details that take place “under the hood.”
The same thing happens when we use computers. Most people use computers to write docu-
ments, send and receive email, surf the web, play music, store images, and play games without
any knowledge of the details that take place to allow those types of applications to work. They
view computers from a logical or user perspective. Computer scientists, programmers, technol-
ogy support staff, and system administrators take a very different view of the computer. They

4 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Figure 1.1: Procedural Abstraction

must know the details of how operating systems work, how network protocols are configured,
and how to code various scripts that control function. They must be able to control the low-level
details that a user simply assumes.
The common point for both of these examples is that the user of the abstraction, sometimes
also called the client, does not need to know the details as long as the user is aware of the way
the interface works. This interface is the way we as users communicate with the underlying
complexities of the implementation. As another example of abstraction, consider the Python
math module. Once we import the module, we can perform computations such as

>>> import math

>>> math.sqrt(16)
4.0
>>>

This is an example of procedural abstraction. We do not necessarily know how the square
root is being calculated, but we know what the function is called and how to use it. If we
perform the import correctly, we can assume that the function will provide us with the correct
results. We know that someone implemented a solution to the square root problem but we only
need to know how to use it. This is sometimes referred to as a “black box” view of a process.
We simply describe the interface: the name of the function, what is needed (the parameters),
and what will be returned. The details are hidden inside (see Figure 1.1).

1.3.1 What Is Programming?

Programming is the process of taking an algorithm and encoding it into a notation, a pro-
gramming language, so that it can be executed by a computer. Although many programming
languages and many different types of computers exist, the important first step is the need to
have the solution. Without an algorithm there can be no program.
Computer science is not the study of programming. Programming, however, is an important
part of what a computer scientist does. Programming is often the way that we create a repre-
sentation for our solutions. Therefore, this language representation and the process of creating
it becomes a fundamental part of the discipline.
Algorithms describe the solution to a problem in terms of the data needed to represent the
problem instance and the set of steps necessary to produce the intended result. Programming
languages must provide a notational way to represent both the process and the data. To this
end, languages provide control constructs and data types.

1.3. What Is Computer Science? 5

Problem Solving with Algorithms and Data Structures, Release 3.0

Control constructs allow algorithmic steps to be represented in a convenient yet unambiguous

way. At a minimum, algorithms require constructs that perform sequential processing, selection
for decision-making, and iteration for repetitive control. As long as the language provides these
basic statements, it can be used for algorithm representation.
All data items in the computer are represented as strings of binary digits. In order to give these
strings meaning, we need to have data types. Data types provide an interpretation for this
binary data so that we can think about the data in terms that make sense with respect to the
problem being solved. These low-level, built-in data types (sometimes called the primitive data
types) provide the building blocks for algorithm development.
For example, most programming languages provide a data type for integers. Strings of binary
digits in the computer’s memory can be interpreted as integers and given the typical meanings
that we commonly associate with integers (e.g. 23, 654, and −19). In addition, a data type also
provides a description of the operations that the data items can participate in. With integers,
operations such as addition, subtraction, and multiplication are common. We have come to
expect that numeric types of data can participate in these arithmetic operations.
The difficulty that often arises for us is the fact that problems and their solutions are very
complex. These simple, language-provided constructs and data types, although certainly suf-
ficient to represent complex solutions, are typically at a disadvantage as we work through the
problem-solving process. We need ways to control this complexity and assist with the creation
of solutions.

1.3.2 Why Study Data Structures and Abstract Data Types?

To manage the complexity of problems and the problem-solving process, computer scientists
use abstractions to allow them to focus on the “big picture” without getting lost in the details.
By creating models of the problem domain, we are able to utilize a better and more efficient
problem-solving process. These models allow us to describe the data that our algorithms will
manipulate in a much more consistent way with respect to the problem itself.
Earlier, we referred to procedural abstraction as a process that hides the details of a particular
function to allow the user or client to view it at a very high level. We now turn our attention to a
similar idea, that of data abstraction. An abstract data type, sometimes called an ADT, is a
logical description of how we view the data and the operations that are allowed without regard
to how they will be implemented. This means that we are concerned only with what the data
is representing and not with how it will eventually be constructed. By providing this level of
abstraction, we are creating an encapsulation around the data. The idea is that by encapsulating
the details of the implementation, we are hiding them from the user’s view. This is called
information hiding.
Figure 1.2 shows a picture of what an abstract data type is and how it operates. The user
interacts with the interface, using the operations that have been specified by the abstract data
type. The abstract data type is the shell that the user interacts with. The implementation is
hidden one level deeper. The user is not concerned with the details of the implementation.
The implementation of an abstract data type, often referred to as a data structure, will require
that we provide a physical view of the data using some collection of programming constructs
and primitive data types. As we discussed earlier, the separation of these two perspectives will

6 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Figure 1.2: Abstract Data Type

allow us to define the complex data models for our problems without giving any indication
as to the details of how the model will actually be built. This provides an implementation-
independent view of the data. Since there will usually be many different ways to implement
an abstract data type, this implementation independence allows the programmer to switch the
details of the implementation without changing the way the user of the data interacts with it.
The user can remain focused on the problem-solving process.

1.3.3 Why Study Algorithms?

Computer scientists learn by experience. We learn by seeing others solve problems and by
solving problems by ourselves. Being exposed to different problem-solving techniques and
seeing how different algorithms are designed helps us to take on the next challenging problem
that we are given. By considering a number of different algorithms, we can begin to develop
pattern recognition so that the next time a similar problem arises, we are better able to solve it.
Algorithms are often quite different from one another. Consider the example of sqrt seen
earlier. It is entirely possible that there are many different ways to implement the details to
compute the square root function. One algorithm may use many fewer resources than another.
One algorithm might take 10 times as long to return the result as the other. We would like to
have some way to compare these two solutions. Even though they both work, one is perhaps
“better” than the other. We might suggest that one is more efficient or that one simply works
faster or uses less memory. As we study algorithms, we can learn analysis techniques that
allow us to compare and contrast solutions based solely on their own characteristics, not the
characteristics of the program or computer used to implement them.
In the worst case scenario, we may have a problem that is intractable, meaning that there is no
algorithm that can solve the problem in a realistic amount of time. It is important to be able
to distinguish between those problems that have solutions, those that do not, and those where
solutions exist but require too much time or other resources to work reasonably.
There will often be trade-offs that we will need to identify and decide upon. As computer
scientists, in addition to our ability to solve problems, we will also need to know and understand

1.3. What Is Computer Science? 7

Problem Solving with Algorithms and Data Structures, Release 3.0

solution evaluation techniques. In the end, there are often many ways to solve a problem.
Finding a solution and then deciding whether it is a good one are tasks that we will do over and
over again.

1.4 Review of Basic Python

In this section, we will review the programming language Python and also provide some more
detailed examples of the ideas from the previous section. If you are new to Python or find that
you need more information about any of the topics presented, we recommend that you consult
a resource such as the Python Language Reference or a Python Tutorial. Our goal here is to
reacquaint you with the language and also reinforce some of the concepts that will be central
to later chapters.
Python is a modern, easy-to-learn, object-oriented programming language. It has a powerful
set of built-in data types and easy-to-use control constructs. Since Python is an interpreted
language, it is most easily reviewed by simply looking at and describing interactive sessions.
You should recall that the interpreter displays the familiar >>> prompt and then evaluates the
Python construct that you provide. For example,

>>> print("Algorithms and Data Structures")

Algorithms and Data Structures
>>>

shows the prompt, the print function, the result, and the next prompt.

1.4.1 Getting Started with Data

We stated above that Python supports the object-oriented programming paradigm. This means
that Python considers data to be the focal point of the problem-solving process. In Python, as
well as in any other object-oriented programming language, we define a class to be a description
of what the data look like (the state) and what the data can do (the behavior). Classes are
analogous to abstract data types because a user of a class only sees the state and behavior of
a data item. Data items are called objects in the object-oriented paradigm. An object is an
instance of a class.

Built-in Atomic Data Types

We will begin our review by considering the atomic data types. Python has two main built-in
numeric classes that implement the integer and floating point data types. These Python classes
are called int and float. The standard arithmetic operations, +, −, *, /, and ** (exponentia-
tion), can be used with parentheses forcing the order of operations away from normal operator
precedence. Other very useful operations are the remainder (modulo) operator, %, and integer
division, //. Note that when two integers are divided, the result is a floating point. The inte-
ger division operator returns the integer portion of the quotient by truncating any fractional part.

8 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Operation Name Operator Explanation

less than < Less than operator
greater than > Greater than operator
less than or equal <= Less than or equal to operator
greater than or equal >= Greater than or equal to operator
equal == Equality operator
not equal =! Not equal operator
logical and and Both operands True for result to be True
logical or or Either operand True for result to be True
logical not not Negates the truth value: False becomes
True, True becomes False

Table 1.1: Relational and Logical Operators

print(2+3*4) #14
print((2+3)*4) #20
print(2**10) #1024
print(6/3) #2.0
print(7/3) #2.33333333333
print(7//3) #2
print(7%3) #1
print(3/6) #0.5
print(3//6) #0
print(3%6) #3
print(2**100) # 1267650600228229401496703205376

The boolean data type, implemented as the Python bool class, will be quite useful for
representing truth values. The possible state values for a boolean object are True and False
with the standard boolean operators, and, or, and not.

>>> True
True
>>> False
False
>>> False or True
True
>>> not (False or True)
False
>>> True and True
True

Boolean data objects are also used as results for comparison operators such as equality (==)
and greater than (>). In addition, relational operators and logical operators can be combined
together to form complex logical questions. Table 1.1 shows the relational and logical
operators with examples shown in the session that follows.

print(5 == 10)
print(10 > 5)

1.4. Review of Basic Python 9

Problem Solving with Algorithms and Data Structures, Release 3.0

Figure 1.3: Variables Hold References to Data Objects

Figure 1.4: Assignment changes the Reference

print((5 >= 1) and (5 <= 10))

Identifiers are used in programming languages as names. In Python, identifiers start with a
letter or an underscore (_), are case sensitive, and can be of any length. Remember that it is
always a good idea to use names that convey meaning so that your program code is easier to
read and understand.
A Python variable is created when a name is used for the first time on the left-hand side of
an assignment statement. Assignment statements provide a way to associate a name with a
value. The variable will hold a reference to a piece of data and not the data itself. Consider the
following session:

>>> the_sum = 0
>>> the_sum
0
>>> the_sum = the_sum + 1
>>> the_sum
1
>>> the_sum = True
>>> the_sum
True

The assignment statement the_sum = 0 creates a variable called the_sum and lets it hold the
reference to the data object 0 (see Figure 1.3). In general, the right-hand side of the assignment
statement is evaluated and a reference to the resulting data object is “assigned” to the name on
the left-hand side. At this point in our example, the type of the variable is integer as that is
the type of the data currently being referred to by “the_sum.” If the type of the data changes
(see Figure 1.4), as shown above with the boolean value True, so does the type of the variable
(the_sum is now of the type boolean). The assignment statement changes the reference being
held by the variable. This is a dynamic characteristic of Python. The same variable can refer to
many different types of data.

10 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Operation Name Operator Explanation

indexing [ ] Access an element of a sequence
concatenation + Combine sequences together
repetition * Concatenate a repeated number of times
membership in Ask whether an item is in a sequence
length len Ask the number of items in the sequence
slicing [ : ] Extract a part of a sequence

Table 1.2: Operations on Any Sequence in Python

Built-in Collection Data Types

In addition to the numeric and boolean classes, Python has a number of very powerful built-
in collection classes. Lists, strings, and tuples are ordered collections that are very similar in
general structure but have specific differences that must be understood for them to be used
properly. Sets and dictionaries are unordered collections.
A list is an ordered collection of zero or more references to Python data objects. Lists are
written as comma-delimited values enclosed in square brackets. The empty list is simply [ ].
Lists are heterogeneous, meaning that the data objects need not all be from the same class and
the collection can be assigned to a variable as below. The following fragment shows a variety
of Python data objects in a list.

>>> [1,3,True,6.5]
[1, 3, True, 6.5]
>>> my_list = [1,3,True,6.5]
>>> my_list
[1, 3, True, 6.5]

Note that when Python evaluates a list, the list itself is returned. However, in order to remember
the list for later processing, its reference needs to be assigned to a variable.
Since lists are considered to be sequentially ordered, they support a number of operations that
can be applied to any Python sequence. Table 1.2 reviews these operations and the following
session gives examples of their use.

Note that the indices for lists (sequences) start counting with 0. The slice operation, my_list[1 :
3], returns a list of items starting with the item indexed by 1 up to but not including the item
indexed by 3.
Sometimes, you will want to initialize a list. This can quickly be accomplished by using
repetition. For example,

>>> my_list = [0] * 6

>>> my_list
[0, 0, 0, 0, 0, 0]

One very important aside relating to the repetition operator is that the result is a repetition
of references to the data objects in the sequence. This can best be seen by considering the

1.4. Review of Basic Python 11

Problem Solving with Algorithms and Data Structures, Release 3.0

Method Name Use Explanation

append a_list.append(item) Adds a new item to the end of a list
insert a_list.insert(i,item) Inserts an item at the 𝑖th position in a list
pop a_list.pop() Removes and returns the last item in a list
pop a_list.pop(i) Removes and returns the 𝑖th item in a list
sort a_list.sort() Modifies a list to be sorted
reverse a_list.reverse() Modifies a list to be in reverse order
del del a_list[i] Deletes the item in the 𝑖th position
index a_list.index(item) Returns the index of the first occurrence of item
count a_list.count(item) Returns the number of occurrences of item
remove a_list.remove(item) Removes the first occurrence of item

Table 1.3: Methods Provided by Lists in Python

following session:

my_list = [1,2,3,4]
A = [my_list]*3
print(A)
my_list[2]=45
print(A)

The variable A holds a collection of three references to the original list called my_list. Note
that a change to one element of my_list shows up in all three occurrences in A.
Lists support a number of methods that will be used to build data structures. Table 1.3 provides
a summary. Examples of their use follow.

my_list = [1024, 3, True, 6.5]

my_list.append(False)
print(my_list)
my_list.insert(2,4.5)
print(my_list)
print(my_list.pop())
print(my_list)
print(my_list.pop(1))
print(my_list)
my_list.pop(2)
print(my_list)
my_list.sort()
print(my_list)
my_list.reverse()
print(my_list)
print(my_list.count(6.5))
print(my_list.index(4.5))
my_list.remove(6.5)
print(my_list)
del my_list[0]
print(my_list)

12 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

You can see that some of the methods, such as pop, return a value and also modify the list.
Others, such as reverse, simply modify the list with no return value. pop will default to
the end of the list but can also remove and return a specific item. The index range starting
from 0 is again used for these methods. You should also notice the familiar “dot” notation
for asking an object to invoke a method. my_list.append(False) can be read as “ask the
object my_list to perform its append method and send it the value False.” Even simple
data objects such as integers can invoke methods in this way.

>>> (54).__add__(21)
75
>>>

In this fragment we are asking the integer object 54 to execute its add method (called __add__
in Python) and passing it 21 as the value to add. The result is the sum, 75. Of course, we usually
write this as 54 + 21. We will say much more about these methods later in this section.
One common Python function that is often discussed in conjunction with lists is the range
function. range produces a range object that represents a sequence of values. By using the
list function, it is possible to see the value of the range object as a list. This is illustrated
below.

>>> range(10)
range(0, 10)
>>> list(range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> range(5,10)
range(5, 10)
>>> list(range(5,10))
[5, 6, 7, 8, 9]
>>> list(range(5,10,2))
[5, 7, 9]
>>> list(range(10,1,-1))
[10, 9, 8, 7, 6, 5, 4, 3, 2]
>>>

The range object represents a sequence of integers. By default, it will start with 0. If you
provide more parameters, it will start and end at particular points and can even skip items. In
our first example, range(10), the sequence starts with 0 and goes up to but does not include
10. In our second example, range(5, 10) starts at 5 and goes up to but not including 10.
range(5, 10, 2) performs similarly but skips by twos (again, 10 is not included).
Strings are sequential collections of zero or more letters, numbers and other symbols. We call
these letters, numbers and other symbols characters. Literal string values are differentiated
from identifiers by using quotation marks (either single or double).

>>> "David"
'David'
>>> my_name = "David"
>>> my_name[3]

1.4. Review of Basic Python 13

Problem Solving with Algorithms and Data Structures, Release 3.0

Method Name Use Explanation

center a_string.center(w) Returns a string centered in a field of size 𝑤
count a_string.count(item) Returns the number of occurrences of item
in the string
ljust a_string.ljust(w) Returns a string left-justified in a field of size
𝑤
lower a_string.lower() Returns a string in all lowercase
rjust a_string.rjust(w) Returns a string right-justified in a field of
size 𝑤
find a_string.find(item) Returns the index of the first occurrence of
item
split a_string.split(s_char) Splits a string into substrings at s_char

Table 1.4: Methods Provided by Strings in Python

'i'
>>> my_name*2
'DavidDavid'
>>> len(my_name)
5
>>>

Since strings are sequences, all of the sequence operations described above work as you would
expect. In addition, strings have a number of methods, some of which are shown in Table 1.4.
For example,

>>> my_name
'David'
>>> my_name.upper()
'DAVID'
>>> my_name.center(10)
' David '
>>> my_name.find('v')
2
>>> my_name.split('v')
['Da', 'id']
>>>

Of these, split will be very useful for processing data. split will take a string and return
a list of strings using the split character as a division point. In the example, v is the division
point. If no division is specified, the split method looks for whitespace characters such as tab,
newline and space.
A major difference between lists and strings is that lists can be modified while strings cannot.
This is referred to as mutability. Lists are mutable; strings are immutable. For example, you
can change an item in a list by using indexing and assignment. With a string that change is not
allowed.

14 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

>>> my_list
[1, 3, True, 6.5]
>>> my_list[0]=2**10
>>> my_list
[1024, 3, True, 6.5]
>>> my_name
'David'
>>> my_name[0]='X'
Traceback (most recent call last):
File "<pyshell#84>", line 1, in <module>
my_name[0]='X'
TypeError: 'str' object does not support item assignment
>>>

Note that the error (or traceback) message displayed above is obtained on a Mac OS X machine.
If you are running the above code snippet on a Windows machine, your error output will more
likely be as follows.
>>> my_name[0]='X'
Traceback (most recent call last):
File "<pyshell#84>", line 1, in -toplevel-
my_name[0]='X'
TypeError: object doesn't support item assignment
>>>

Depending on your operating system, or version of Python, the output may slightly vary. How-
ever it will still indicate where and what the error is. You may want to experiment for yourself
and get acquainted with the error message for easier and faster debugging. For the remainder
of this work, we will only display the Mac OS X error messages.
Tuples are very similar to lists in that they are heterogeneous sequences of data. The difference
is that a tuple is immutable, like a string. A tuple cannot be changed. Tuples are written as
comma-delimited values enclosed in parentheses. As sequences, they can use any operation
described above. For example,

>>> my_tuple = (2,True,4.96)

>>> my_tuple
(2, True, 4.96)
>>> len(my_tuple)
3
>>> my_tuple[0]
2
>>> my_tuple * 3
(2, True, 4.96, 2, True, 4.96, 2, True, 4.96)
>>> my_tuple[0:2]
(2, True)
>>>

However, if you try to change an item in a tuple, you will get an error. Note that the error
message provides location and reason for the problem.

1.4. Review of Basic Python 15

Problem Solving with Algorithms and Data Structures, Release 3.0

Operator Use Explanation

in x.in(set) Set membership
len len(set) Returns the cardinality (i.e. the length) of the set
| set1 | set2 Returns a new set with all elements from both sets
& set1 & set2 Returns a new set with only the elements common to both sets
- set1 - set2 Returns a new set with all items from the first set not in second
<= set1 <= set2 Asks whether all elements of the first set are in the second

Table 1.5: Operations on a Set in Python

>>> my_tuple[1]=False
Traceback (most recent call last):
File "<pyshell#137>", line 1, in <module>
my_tuple[1]=False
TypeError: 'tuple' object does not support item assignment
>>>

A set is an unordered collection of zero or more immutable Python data objects. Sets do not
allow duplicates and are written as comma-delimited values enclosed in curly braces. The
empty set is represented by set(). Sets are heterogeneous, and the collection can be assigned
to a variable as below.

>>> {3,6,"cat",4.5,False}
{False, 4.5, 3, 6, 'cat'}
>>> my_set = {3,6,"cat",4.5,False}
>>> my_set
{False, 3, 4.5, 6, 'cat'}
>>>

Even though sets are not considered to be sequential, they do support a few of the familiar
operations presented earlier. Table 1.5 reviews these operations and the following session gives
examples of their use.

>>> my_set
{False, 3, 4.5, 6, 'cat'}
>>> len(my_set)
5
>>> False in my_set
True
>>> "dog" in my_set
False
>>>

Sets support a number of methods that should be familiar to those who have worked with them
in a mathematics setting. Table 1.6 provides a summary. Examples of their use follow. Note
that union, intersection, issubset, and difference all have operators that can be
used as well.

16 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Method Name Use Explanation

union set1.union(set2) Returns a new set with all elements from
both sets
intersection set1.intersection(set2) Returns a new set with only the elements
common to both sets
difference set1.difference(set2) Returns a new set with all items from first set
not in second
issubset set1.issubset(set2) Asks whether all elements of one set are in
the other
add set.add(item) Adds item to the set
remove set.remove(item) Removes item from the set
pop set.pop() Removes an arbitrary element from the set
clear set.clear() Removes all elements from the set

Table 1.6: Methods Provided by Sets in Python

>>> my_set
{False, 3, 4.5, 6, 'cat'}
>>> your_set = {99,3,100}
>>> my_set.union(your_set)
{False, 3, 4.5, 6, 99, 'cat', 100}
>>> my_set | your_set
{False, 3, 4.5, 6, 99, 'cat', 100}
>>> my_set.intersection(your_set)
{3}
>>> my_set & your_set
{3}
>>> my_set.difference(your_set)
{False, 4.5, 6, 'cat'}
>>> my_set - your_set
{False, 4.5, 6, 'cat'}
>>> {3,100}.issubset(your_set)
True
>>> {3,100} <= your_set
True
>>> my_set.add("house")
>>> my_set
{False, 3, 4.5, 6, 'house', 'cat'}
>>> my_set.remove(4.5)
>>> my_set
{False, 3, 6, 'house', 'cat'}
>>> my_set.pop()
False
>>> my_set
{3, 6, 'house', 'cat'}
>>> my_set.clear()
>>> my_set
set()
>>>

1.4. Review of Basic Python 17

Problem Solving with Algorithms and Data Structures, Release 3.0

Operator Use Explanation

[] my_dict[k] Returns the value associated with 𝑘, otherwise its an error
in key in my_dict Returns True if key is in the dictionary, False otherwise
del del my_dict[key] Removes the entry from the dictionary

Table 1.7: Operators Provided by Dictionaries in Python

Our final Python collection is an unordered structure called a dictionary. Dictionaries are
collections of associated pairs of items where each pair consists of a key and a value. This
key-value pair is typically written as key:value. Dictionaries are written as comma-delimited
key:value pairs enclosed in curly braces. For example,

>>> capitals = {'Iowa':'DesMoines','Wisconsin':'Madison'}

>>> capitals
{'Wisconsin': 'Madison', 'Iowa': 'DesMoines'}
>>>

We can manipulate a dictionary by accessing a value via its key or by adding another key-value
pair. The syntax for access looks much like a sequence access except that instead of using the
index of the item we use the key value. To add a new value is similar.

capitals = {'Iowa':'DesMoines','Wisconsin':'Madison'}
print(capitals['Iowa'])
capitals['Utah']='SaltLakeCity'
print(capitals)
capitals['California']='Sacramento'
print(len(capitals))
for k in capitals:
print(capitals[k]," is the capital of ", k)

It is important to note that the dictionary is maintained in no particular order with respect to the
keys. The first pair added ('Utah': 'SaltLakeCity') was placed first in the dictionary
and the second pair added ('California': 'Sacramento') was placed last. The place-
ment of a key is dependent on the idea of “hashing,” which will be explained in more detail
in Chapter 4. We also show the length function performing the same role as with previous
collections.
Dictionaries have both methods and operators. Table 1.7 and Table 1.8 describe them, and the
session shows them in action. The keys, values, and items methods all return objects that
contain the values of interest. You can use the list function to convert them to lists. You
will also see that there are two variations on the get method. If the key is not present in the
dictionary, get will return None. However, a second, optional parameter can specify a return
value instead.

>>> phone_ext={'david':1410, 'brad':1137}

>>> phone_ext
{'brad': 1137, 'david': 1410}
>>> phone_ext.keys() # Returns the keys of the dictionary phone_ext

18 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Method Name Use Explanation

keys my_dict.keys() Returns the keys of the dictionary in a dict_keys
object
values my_dict.values() Returns the values of the dictionary in a
dict_values object
items my_dict.items() Returns the key-value pairs in a dict_items ob-
ject
get my_dict.get(k) Returns the value associated with 𝑘, None other-
wise
get my_dict.get(k,alt) Returns the value associated with 𝑘, 𝑎𝑙𝑡 otherwise

Table 1.8: Methods Provided by Dictionaries in Python

dict_keys(['brad', 'david'])
>>> list(phone_ext.keys())
['brad', 'david']
>>> "brad" in phone_ext
>>> True
>>> 1137 in phone_ext
>>> False # 1137 is not a key in phone_ext
>>> phone_ext.values() # Returns the values of the dictionary
phone_ext
dict_values([1137, 1410])
>>> list(phone_ext.values())
[1137, 1410]
>>> phone_ext.items()
dict_items([('brad', 1137), ('david', 1410)])
>>> list(phone_ext.items())
[('brad', 1137), ('david', 1410)]
>>> phone_ext.get("kent")
>>> phone_ext.get("kent","NO ENTRY")
'NO ENTRY'
>>> del phone_ext["david"]
>>> phone_ext
{'brad': 1137}
>>>

1.4.2 Input and Output

We often have a need to interact with users, either to get data or to provide some sort of result.
Most programs today use a dialog box as a way of asking the user to provide some type of
input. While Python does have a way to create dialog boxes, there is a much simpler function
that we can use. Python provides us with a function that allows us to ask a user to enter some
data and returns a reference to the data in the form of a string. The function is called input.
Python’s input function takes a single parameter that is a string. This string is often called
the prompt because it contains some helpful text prompting the user to enter something. For
example, you might call input as follows:

1.4. Review of Basic Python 19

Problem Solving with Algorithms and Data Structures, Release 3.0

user_name = input('Please enter your name: ')

Now whatever the user types after the prompt will be stored in the user_name variable.
Using the input function, we can easily write instructions that will prompt the user to enter
data and then incorporate that data into further processing. For example, in the following two
statements, the first asks the user for their name and the second prints the result of some simple
processing based on the string that is provided.

user_name = input("Please enter your name ")

print("Your name in all capitals is",user_name.upper(),
"and has length", len(user_name))

It is important to note that the value returned from the input function will be a string
representing the exact characters that were entered after the prompt. If you want this string
interpreted as another type, you must provide the type conversion explicitly. In the statements
below, the string that is entered by the user is converted to a float so that it can be used in
further arithmetic processing.

user_radius = input("Please enter the radius of the circle ")

radius = float(user_radius)
diameter = 2 * radius

String Formatting

We have already seen that the print function provides a very simple way to output values
from a Python program. print takes zero or more parameters and displays them using a single
blank as the default separator. It is possible to change the separator character by setting the
sep argument. In addition, each print ends with a newline character by default. This behavior
can be changed by setting the end argument. These variations are shown in the following
session:

>>> print("Hello")
Hello
>>> print("Hello","World")
Hello World
>>> print("Hello","World", sep="***")
Hello***World
>>> print("Hello","World", end="***")
Hello World***
>>> print("Hello", end="***"); print("World")
Hello***World
>>>

It is often useful to have more control over the look of your output. Fortunately, Python
provides us with an alternative called formatted strings. A formatted string is a template in

20 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Character Output Format

d,i Integer
u Unsigned Integer
f Floating point as m.ddddd
e Floating point as m.ddddde+/-xx
E Floating point as m.dddddE+/-xx
g Use %e for exponents less than −4 or greater than +5, otherwise us %f
c Single character
s String, or any Python data object that can be converted to a string by using the str
function
% Insert a literal % character

Table 1.9: String Formatting Conversion Characters

which words or spaces that will remain constant are combined with placeholders for variables
that will be inserted into the string. For example, the statement

print(name, "is", age, "years old.")

contains the words is and years old, but the name and the age will change depending on
the variable values at the time of execution. Using a formatted string, we write the previous
statement as

print("%s is %d years old." % (name, age))

This simple example illustrates a new string expression. The % operator is a string operator
called the format operator. The left side of the expression holds the template or format string,
and the right side holds a collection of values that will be substituted into the format string.
Note that the number of values in the collection on the right side corresponds with the number
of % characters in the format string. Values are taken in order, left to right from the collection
and inserted into the format string.
Let’s look at both sides of this formatting expression in more detail. The format string may
contain one or more conversion specifications. A conversion character tells the format opera-
tor what type of value is going to be inserted into that position in the string. In the example
above, the %s specifies a string, while the %d specifies an integer. Other possible type spec-
ifications include i, u, f, e, g, c, or %. Table 1.9 summarizes all of the various type
specifications.
In addition to the format character, you can also include a format modifier between the % and
the format character. Format modifiers may be used to left-justify or right-justify the value
with a specified field width. Modifiers can also be used to specify the field width along with a
number of digits after the decimal point. Table 1.10 explains these format modifiers.
The right side of the format operator is a collection of values that will be inserted into the
format string. The collection will be either a tuple or a dictionary. If the collection is a tuple,
the values are inserted in order of position. That is, the first element in the tuple corresponds
to the first format character in the format string. If the collection is a dictionary, the values
are inserted according to their keys. In this case all format characters must use the (name)

1.4. Review of Basic Python 21

Problem Solving with Algorithms and Data Structures, Release 3.0

Modifier Example Description

number %20d Put the value in a field width of 20
- %-20d Put the value in a field 20 characters wide, left-justified
+ %+20d Put the value in a field 20 characters wide, right-justified
0 %020d Put the value in a field 20 characters wide, fill in with leading zeros
. %20.2f Put the value in a field 20 characters wide with 2 characters to the right
of the decimal point.
(name) %(name)d Get the value from the supplied dictionary using name as the key.

Table 1.10: Additional formatting options

modifier to specify the name of the key.

>>> price = 24
>>> item = "banana"
>>> print("The %s costs %d cents"%(item,price))
The banana costs 24 cents
>>> print("The %+10s costs %5.2f cents"%(item,price))
The banana costs 24.00 cents
>>> print("The %+10s costs %10.2f cents"%(item,price))
The banana costs 24.00 cents
>>> item_dict = {"item":"banana","cost":24}
>>> print("The %(item)s costs %(cost)7.1f cents"%item_dict)
The banana costs 24.0 cents
>>>

In addition to format strings that use format characters and format modifiers, Python strings
also include a format method that can be used in conjunction with a new Formatter class
to implement complex string formatting. More about these features can be found in the Python
library reference manual.

1.4.3 Control Structures

As we noted earlier, algorithms require two important control structures: iteration and selection.
Both of these are supported by Python in various forms. The programmer can choose the
statement that is most useful for the given circumstance.
For iteration, Python provides a standard while statement and a very powerful for statement.
The while statement repeats a body of code as long as a condition is true. For example,

>>> counter = 1
>>> while counter <= 5:
print("Hello, world")
counter = counter + 1

Hello, world
Hello, world

22 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Hello, world
Hello, world
Hello, world
>>>

prints out the phrase “Hello, world” five times. The condition on the while statement is eval-
uated at the start of each repetition. If the condition is True, the body of the statement will
execute. It is easy to see the structure of a Python while statement due to the mandatory
indentation pattern that the language enforces.
The while statement is a very general purpose iterative structure that we will use in a number
of different algorithms. In many cases, a compound condition will control the iteration. A
fragment such as

while counter <= 10 and not done:

...

would cause the body of the statement to be executed only in the case where both parts of the
condition are satisfied. The value of the variable counter would need to be less than or equal to
10 and the value of the variable done would need to be False (not False is True) so that
True and True results in True.
Even though this type of construct is very useful in a wide variety of situations, another
iterative structure, the for statement, can be used in conjunction with many of the Python
collections. The for statement can be used to iterate over the members of a collection, so long
as the collection is a sequence. So, for example,

>>> for item in [1,3,6,2,5]:

print(item)

1
3
6
2
5
>>>

assigns the variable item to be each successive value in the list [1, 3, 6, 2, 5]. The body of the
iteration is then executed. This works for any collection that is a sequence (lists, tuples, and
strings).
A common use of the for statement is to implement definite iteration over a range of values.
The statement

>>> for item in range(5):

print(item ** 2)

0
1

1.4. Review of Basic Python 23

Problem Solving with Algorithms and Data Structures, Release 3.0

4
9
16
>>>

will perform the print function five times. The range function will return a range object
representing the sequence 0, 1, 2, 3, 4 and each value will be assigned to the variable item.
This value is then squared and printed.
The other very useful version of this iteration structure is used to process each character of a
string. The following code fragment iterates over a list of strings and for each string processes
each character by appending it to a list. The result is a list of all the letters in all of the words.

word_list = ['cat','dog','rabbit']
letter_list = [ ]
for a_word in word_list:
for a_letter in a_word:
letter_list.append(a_letter)
print(letter_list)

Selection statements allow programmers to ask questions and then, based on the result, perform
different actions. Most programming languages provide two versions of this useful construct:
the ifelse and the if. A simple example of a binary selection uses the ifelse statement.

if n < 0:
print("Sorry, value is negative")
else:
print(math.sqrt(n))

In this example, the object referred to by n is checked to see if it is less than zero. If it is, a
message is printed stating that it is negative. If it is not, the statement performs the else clause
and computes the square root.
Selection constructs, as with any control construct, can be nested so that the result of one
question helps decide whether to ask the next. For example, assume that score is a variable
holding a reference to a score for a computer science test.

if score >= 90:

print('A')
else:
if score >= 80:
print('B')
else:
if score >= 70:
print('C')
else:
if score >= 60:
print('D')
else:

24 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

print('F')

Python also has a single way selection construct, the if statement. With this statement, if the
condition is true, an action is performed. In the case where the condition is false, processing
simply continues on to the next statement after the if. For example, the following fragment
will first check to see if the value of a variable n is negative. If it is, then it is modified by the
absolute value function. Regardless, the next action is to compute the square root.

if n < 0:
n = abs(n)
print(math.sqrt(n))

Returning to lists, there is an alternative method for creating a list that uses iteration
and selection constructs. The is known as a list comprehension. A list comprehension
allows you to easily create a list based on some processing or selection criteria. For exam-
ple, if we would like to create a list of the first 10 perfect squares, we could use a for statement:

>>> sq_list = []
>>> for x in range(1, 11):
sq_list.append(x * x)

>>> sq_list
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
>>>

Using a list comprehension, we can do this in one step as

>>> sq_list = [x * x for x in range(1, 11)]

>>> sq_list
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
>>>

The variable x takes on the values 1 through 10 as specified by the for construct. The value
of x * x is then computed and added to the list that is being constructed. The general syntax
for a list comprehension also allows a selection criteria to be added so that only certain items
get added. For example,

>>> sq_list = [x * x for x in range(1, 11) if x % 2 != 0]

>>> sq_list
[1, 9, 25, 49, 81]
>>>

This list comprehension constructed a list that only contained the squares of the odd numbers
in the range from 1 to 10. Any sequence that supports iteration can be used within a list
comprehension to construct a new list.

>>>[ch.upper() for ch in 'comprehension' if ch not in 'aeiou']

1.4. Review of Basic Python 25

Problem Solving with Algorithms and Data Structures, Release 3.0

['C', 'M', 'P', 'R', 'H', 'N', 'S', 'N']

>>>

Self Check

Test your understanding of what we have covered so far by trying the following two exercises.
Use the code below, seen earlier in this subsection.
word_list = ['cat','dog','rabbit']
letter_list = [ ]
for a_word in word_list:
for a_letter in a_word:
letter_list.append(a_letter)
print(letter_list)

1. Modify the given code so that the final list only contains a single copy of each letter.
# the answer is: ['c', 'a', 't', 'd', 'o', 'g', 'r', 'b', 'i']

2. Redo the given code using list comprehensions. For an extra challenge, see if you can
figure out how to remove the duplicates.
# the answer is: ['c', 'a', 't', 'd', 'o', 'g', 'r', 'a',
'b', 'b', 'i', 't']

1.4.4 Exception Handling

There are two types of errors that typically occur when writing programs. The first, known
as a syntax error, simply means that the programmer has made a mistake in the structure of
a statement or expression. For example, it is incorrect to write a for statement and forget the
colon.

>>> for i in range(10)

SyntaxError: invalid syntax
>>>

In this case, the Python interpreter has found that it cannot complete the processing of this
instruction since it does not conform to the rules of the language. Syntax errors are usually
more frequent when you are first learning a language.
The other type of error, known as a logic error, denotes a situation where the program executes
but gives the wrong result. This can be due to an error in the underlying algorithm or an error in
your translation of that algorithm. In some cases, logic errors lead to very bad situations such
as trying to divide by zero or trying to access an item in a list where the index of the item is
outside the bounds of the list. In this case, the logic error leads to a runtime error that causes
the program to terminate. These types of runtime errors are typically called exceptions.

26 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Most of the time, beginning programmers simply think of exceptions as fatal runtime errors
that cause the end of execution. However, most programming languages provide a way to deal
with these errors that will allow the programmer to have some type of intervention if they so
choose. In addition, programmers can create their own exceptions if they detect a situation in
the program execution that warrants it.
When an exception occurs, we say that it has been “raised.” You can “handle” the exception
that has been raised by using a try statement. For example, consider the following session
that asks the user for an integer and then calls the square root function from the math library.
If the user enters a value that is greater than or equal to 0, the print will show the square root.
However, if the user enters a negative value, the square root function will report a ValueError
exception.

>>> a_number = int(input("Please enter an integer "))

Please enter an integer -23
>>> print(math.sqrt(a_number))
Traceback (most recent call last):
File "<pyshell#102>", line 1, in <module>
print(math.sqrt(a_number))
ValueError: math domain error
>>>

We can handle this exception by calling the print function from within a try block. A
corresponding except block “catches” the exception and prints a message back to the user in
the event that an exception occurs. For example:

>>> try:
print(math.sqrt(a_number))
except:
print("Bad Value for square root")
print("Using absolute value instead")
print(math.sqrt(abs(a_number)))

Bad Value for square root

Using absolute value instead
4.795831523312719
>>>

will catch the fact that an exception is raised by sqrt and will instead print the messages back
to the user and use the absolute value to be sure that we are taking the square root of a non-
negative number. This means that the program will not terminate but instead will continue on
to the next statements.
It is also possible for a programmer to cause a runtime exception by using the raise statement.
For example, instead of calling the square root function with a negative number, we could have
checked the value first and then raised our own exception. The code fragment below shows
the result of creating a new RuntimeError exception. Note that the program would still
terminate but now the exception that caused the termination is something explicitly created by

1.4. Review of Basic Python 27

Problem Solving with Algorithms and Data Structures, Release 3.0

the programmer.

>>> if a_number < 0:

... raise RuntimeError("You can't use a negative number")
... else:
... print(math.sqrt(a_number))
...
Traceback (most recent call last):
File "<pyshell#20>", line 2, in <module>
raise RuntimeError("You can't use a negative number")
RuntimeError: You can't use a negative number
>>>

There are many kinds of exceptions that can be raised in addition to the RuntimeError shown
above. See the Python reference manual for a list of all the available exception types and for
how to create your own.

1.4.5 Defining Functions

The earlier example of procedural abstraction called upon a Python function called sqrt
from the math module to compute the square root. In general, we can hide the details of
any computation by defining a function. A function definition requires a name, a group of
parameters, and a body. It may also explicitly return a value. For example, the simple function
defined below returns the square of the value you pass into it.

>>> def square(n):

... return n ** 2
...
>>> square(3)
9
>>> square(square(3))
81
>>>

The syntax for this function definition includes the name, square, and a parenthesized list
of formal parameters. For this function, n is the only formal parameter, which suggests that
square needs only one piece of data to do its work. The details, hidden “inside the box,”
simply compute the result of n ** 2 and return it. We can invoke or call the square function
by asking the Python environment to evaluate it, passing an actual parameter value, in this
case, 3. Note that the call to square returns an integer that can in turn be passed to another
invocation.
We could implement our own square root function by using a well-known technique called
“Newton’s Method.” Newton’s Method for approximating square roots performs an iterative
computation that converges on the correct value. The equation
1 𝑜𝑙𝑑_𝑔𝑢𝑒𝑠𝑠 + 𝑛
𝑛𝑒𝑤_𝑔𝑢𝑒𝑠𝑠 = *( )
2 𝑜𝑙𝑑_𝑔𝑢𝑒𝑠𝑠

28 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

takes a value 𝑛 and repeatedly guesses the square root by making each new_guess the
old_guess in the subsequent iteration. The initial guess used here is 𝑛2 . Listing 1.1 shows a
function definition that accepts a value 𝑛 and returns the square root of 𝑛 after making 20
guesses. Again, the details of Newton’s Method are hidden inside the function definition and
the user does not have to know anything about the implementation to use the function for its
intended purpose. Listing 1.1 also shows the use of the # character as a comment marker. Any
characters that follow the # on a line are ignored.

Listing 1.1: square_root Function

def square_root(n):
root = n / 2 #initial guess will be 1/2 of n
for k in range(20):
root = (1 / 2) * (root + (n / root))

return root

>>>square_root(9)
3.0
>>>square_root(4563)
67.549981495186216
>>>

Self Check

Here is a self check that really covers everything so far. You may have heard of the infinite
monkey theorem? The theorem states that a monkey hitting keys at random on a typewriter
keyboard for an infinite amount of time will almost surely type a given text, such as the com-
plete works of William Shakespeare. Well, suppose we replace a monkey with a Python func-
tion. How long do you think it would take for a Python function to generate just one sentence
of Shakespeare? The sentence we’ll shoot for is: “methinks it is like a weasel”
You are not going to want to run this one in the browser, so fire up your favorite Python IDE. The
way we will simulate this is to write a function that generates a string that is 27 characters long
by choosing random letters from the 26 letters in the alphabet plus the space. We will write
another function that will score each generated string by comparing the randomly generated
string to the goal.
A third function will repeatedly call generate and score, then if 100% of the letters are correct
we are done. If the letters are not correct then we will generate a whole new string. To make
it easier to follow your program’s progress this third function should print out the best string
generated so far and its score every 1000 tries.

Self Check Challenge

See if you can improve upon the program in the self check by keeping letters that are correct
and only modifying one character in the best string so far. This is a type of algorithm in the

1.4. Review of Basic Python 29

Problem Solving with Algorithms and Data Structures, Release 3.0

class of “hill climbing” algorithms, that is we only keep the result if it is better than the previous
one.

1.4.6 Object-Oriented Programming in Python: Defining Classes

We stated earlier that Python is an object-oriented programming language. So far, we have

used a number of built-in classes to show examples of data and control structures. One of the
most powerful features in an object-oriented programming language is the ability to allow a
programmer (problem solver) to create new classes that model data that is needed to solve the
problem.
Remember that we use abstract data types to provide the logical description of what a data
object looks like (its state) and what it can do (its methods). By building a class that implements
an abstract data type, a programmer can take advantage of the abstraction process and at the
same time provide the details necessary to actually use the abstraction in a program. Whenever
we want to implement an abstract data type, we will do so with a new class.

A Fraction Class

A very common example to show the details of implementing a user-defined class is to construct
a class to implement the abstract data type Fraction. We have already seen that Python
provides a number of numeric classes for our use. There are times, however, that it would be
most appropriate to be able to create data objects that “look like” fractions.
A fraction such as 35 consists of two parts. The top value, known as the numerator, can be
any integer. The bottom value, called the denominator, can be any integer greater than 0 (neg-
ative fractions have a negative numerator). Although it is possible to create a floating point
approximation for any fraction, in this case we would like to represent the fraction as an exact
value.
The operations for the Fraction type will allow a Fraction data object to behave like any
other numeric value. We need to be able to add, subtract, multiply, and divide fractions. We
also want to be able to show fractions using the standard “slash” form, for example 35 . In
addition, all fraction methods should return results in their lowest terms so that no matter what
computation is performed, we always end up with the most common form.
In Python, we define a new class by providing a name and a set of method definitions that are
syntactically similar to function definitions. For this example,

class Fraction:

#the methods go here

provides the framework for us to define the methods. The first method that all classes should
provide is the constructor. The constructor defines the way in which data objects are created.
To create a Fraction object, we will need to provide two pieces of data, the numerator
and the denominator. In Python, the constructor method is always called __init__(two

30 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Figure 1.5: An Instance of the Fraction class

underscores before and after init) and is shown in Listing 1.2.

Listing 1.2: Fraction Class and its constructor

class Fraction:

def __init__(self,top,bottom):

self.num = top
self.den = bottom

Notice that the formal parameter list contains three items (self, top, bottom). self is a
special parameter that will always be used as a reference back to the object itself. It must
always be the first formal parameter; however, it will never be given an actual parameter value
upon invocation. As described earlier, fractions require two pieces of state data, the numerator
and the denominator. The notation self.num in the constructor defines the fraction object
to have an internal data object called num as part of its state. Likewise, self.den creates
the denominator. The values of the two formal parameters are initially assigned to the state,
allowing the new fraction object to know its starting value.
To create an instance of the Fraction class, we must invoke the constructor. This happens
by using the name of the class and passing actual values for the necessary state (note that we
never directly invoke __init__). For example,

my_fraction = Fraction(3,5)

3
creates an object called my_fraction representing the fraction 5
(three-fifths). Figure 1.5
shows this object as it is now implemented.
The next thing we need to do is implement the behavior that the abstract data type requires. To
begin, consider what happens when we try to print a Fraction object.

>>> my_f = Fraction(3, 5)

>>> print(my_f)

1.4. Review of Basic Python 31

Problem Solving with Algorithms and Data Structures, Release 3.0

<__main__.Fraction object at 0x409b1acc>

The Fraction object, my_f, does not know how to respond to this request to print. The print
function requires that the object convert itself into a string so that the string can be written to the
output. The only choice my_f has is to show the actual reference that is stored in the variable
(the address itself). This is not what we want.
There are two ways we can solve this problem. One is to define a method called show that will
allow the Fraction object to print itself as a string. We can implement this method as shown
in Listing 1.3. If we create a Fraction object as before, we can ask it to show itself, in other
words, print itself in the proper format. Unfortunately, this does not work in general. In order
to make printing work properly, we need to tell the Fraction class how to convert itself into
a string. This is what the print function needs in order to do its job.

Listing 1.3: Show Function

def show(self):
print(self.num, "/", self.den)
>>> my_f = Fraction(3, 5)
>>> my_f.show()
3 / 5
>>> print(my_f)
<__main__.Fraction object at 0x40bce9ac>
>>>

In Python, all classes have a set of standard methods that are provided but may not work prop-
erly. One of these, __str__, is the method to convert an object into a string. The default
implementation for this method is to return the instance address string as we have already seen.
What we need to do is provide a “better” implementation for this method. We will say that this
implementation overrides the previous one, or that it redefines the method’s behavior.
To do this, we simply define a method with the name __str__ and give it a new implemen-
tation as shown in Listing 1.4. This definition does not need any other information except the
special parameter self. In turn, the method will build a string representation by converting
each piece of internal state data to a string and then placing a / character in between the strings
using string concatenation. The resulting string will be returned any time a Fraction object
is asked to convert itself to a string. Notice the various ways that this function is used.

Listing 1.4: Standard Method

def __str__ (self):
return str(self.num) + "/" + str(self.den)

>>> my_f = Fraction(3, 5)

>>> print(my_f)
3/5
>>> print("I ate", my_f, "of the pizza")
I ate 3/5 of the pizza
>>> my_f.__str__()

32 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

'3/5'
>>> str(my_f)
'3/5'
>>>

We can override many other methods for our new Fraction class. Some of the most
important of these are the basic arithmetic operations. We would like to be able to create two
Fraction objects and then add them together using the standard “+” notation. At this point,
if we try to add two fractions, we get the following:

>>> f1 = Fraction(1,4)
>>> f2 = Fraction(1,2)
>>> f1 + f2
Traceback (most recent call last):
File "<pyshell#26>", line 1, in <module>
f1 + f2
TypeError: unsupported operand type(s) for +: 'Fraction' and
'Fraction'
>>>

If you look closely at the error, you see that the problem is that the “+” operator does not
understand the Fraction operands. We can fix this by providing the Fraction class with a
method that overrides the addition method. In Python, this method is called __add__ and it
requires two parameters. The first, self, is always needed, and the second represents the other
operand in the expression. For example,

f1.__add__(f2)

would ask the Fraction object f1 to add the Fraction object f2 to itself. This can be
written in the standard notation, f1 + f2.
Two fractions must have the same denominator to be added. The easiest way to make sure they
have the same denominator is to simply use the product of the two denominators as a common
denominator so that
𝑎 𝑐 𝑎𝑑 𝑐𝑏 𝑎𝑑 + 𝑐𝑏
+ = + =
𝑏 𝑑 𝑏𝑑 𝑏𝑑 𝑏𝑑
The implementation is shown in Listing 1.5. The addition function returns a new Fraction
object with the numerator and denominator of the sum. We can use this method by writing
a standard arithmetic expression involving fractions, assigning the result of the addition, and
then printing our result.

Listing 1.5: Adding Fractions

def __add__(self, other_fraction):

new_num = self.num*other_fraction.den +
self.den*other_fraction.num
new_den = self.den * other_fraction.den

1.4. Review of Basic Python 33

Problem Solving with Algorithms and Data Structures, Release 3.0

return Fraction(new_num, new_den)

>>> f1 = Fraction(1, 4)
>>> f2 = Fraction(1, 2)
>>> f3 = f1 + f2
>>> print(f3)
6/8
>>>

The addition method works as we desire, but one thing could be better. Note that 68 is the correct
result ( 14 + 12) but that it is not in the “lowest terms” representation. The best representation
would be 34 . In order to be sure that our results are always in the lowest terms, we need a helper
function that knows how to reduce fractions. This function will need to look for the greatest
common divisor, or GCD. We can then divide the numerator and the denominator by the GCD
and the result will be reduced to lowest terms.
The best-known algorithm for finding a greatest common divisor is Euclid’s Algorithm.
Euclid’s Algorithm states that the greatest common divisor of two integers 𝑚 and 𝑛 is 𝑛 if
𝑛 divides 𝑚 evenly. However, if 𝑛 does not divide 𝑚 evenly, then the answer is the greatest
common divisor of 𝑛 and the remainder of 𝑚 divided by 𝑛. We will simply provide an iterative
implementation here. Note that this implementation of the GCD algorithm only works when
the denominator is positive. This is acceptable for our fraction class because we have said that
a negative fraction will be represented by a negative numerator.

def gcd(m, n):

while m % n != 0:
old_m = m
old_n = n

m = old_n
n = old_m % old_n
return n

print(gcd(20, 10))

Now we can use this function to help reduce any fraction. To put a fraction in lowest terms,
we will divide the numerator and the denominator by their greatest common divisor. So, for
the fraction 68 , the greatest common divisor is 2. Dividing the top and the bottom by 2 creates
a new fraction, 34 (see Listing 1.6).

Listing 1.6: Lowest Term Fractions

def __add__(self, other_fraction):
new_num = self.num*other_fraction.den +
self.den*other_fraction.num
new_den = self.den * other_fraction.den
common = gcd(new_num, new_den)
return Fraction(new_num // common, new_den // common)

34 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

Figure 1.6: An Instance of the Fraction Class with Two Methods

>>> f1 = Fraction(1, 4)
>>> f2 = Fraction(1, 2)
>>> f3 = f1 + f2
>>> print(f3)
3/4
>>>

Our Fraction object now has two very useful methods and looks like Figure 1.6. An addi-
tional group of methods that we need to include in our example Fraction class will allow
two fractions to compare themselves to one another. Assume we have two Fraction objects, f1
and f2. f1 == f2 will only be True if they are references to the same object. Two different
objects with the same numerators and denominators would not be equal under this implemen-
tation. This is called shallow equality (see Figure 1.7).
We can create deep equality (see Figure 1.7) – equality by the same value, not the same ref-
erence – by overriding the __eq__ method. The __eq__ method is another standard method
available in any class. The __eq__ method compares two objects and returns True if their
values are the same, False otherwise.
In the Fraction class, we can implement the __eq__ method by again putting the two
fractions in common terms and then comparing the numerators (see Listing 1.7). It is
important to note that there are other relational operators that can be overridden. For example,
the __le__ method provides the less than or equal functionality.

Listing 1.7: Checking If Two Fractions are Equal

def __eq__(self, other):
first_num = self.num * other.den
second_num = other.num * self.den

return first_num == second_num

The complete Fraction class, up to this point, is shown below. We leave the remaining
arithmetic and relational methods as exercises.

1.4. Review of Basic Python 35

Problem Solving with Algorithms and Data Structures, Release 3.0

Figure 1.7: Shallow Equality Versus Deep Equality

36 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

# gcd function
def gcd(m, n):
while m % n != 0:
old_m = m
old_n = n

m = old_n
n = old_m % old_n
return n

# Fraction class
# Implements: addition and equality
# To do: multiplication, division, subtraction and comparison
operators (< , >)
class Fraction:
def __init__(self, top, bottom):
self.num = top
self.den = bottom

def __str__(self):
return str(self.num) + "/" + str(self.den)

def show(self):
print(self.num, "/", self.den)

def __add__(self, other_fraction):

new_num = self.num * other_fraction.den + \
self.den * other_fraction.num
new_den = self.den * other_fraction.den
common = gcd(new_num, new_den)
return Fraction(new_num // common, new_den // common)

def __eq__(self, other):

first_num = self.num * other.den
second_num = other.num * self.den

return first_num == second_num

x = Fraction(1, 2)
y = Fraction(2, 3)
print(x + y)
print(x == y)

Self Check

To make sure you understand how operators are implemented in Python classes, and how to
properly write methods, write some methods to implement *, /, and −. Also implement com-
parison operators > and <.

1.4. Review of Basic Python 37

Problem Solving with Algorithms and Data Structures, Release 3.0

1.5 Summary

• Computer science is the study of problem solving.

• Computer science uses abstraction as a tool for representing both processes and data.
• Abstract data types allow programmers to manage the complexity of a problem domain
by hiding the details of the data.
• Python is a powerful, yet easy-to-use, object-oriented language.
• Lists, tuples, and strings are built in Python sequential collections.
• Dictionaries and sets are nonsequential collections of data.
• Classes allow programmers to implement abstract data types.
• Programmers can override standard methods as well as create new methods.
• A class constructor should always invoke the constructor of its parent before continuing
on with its own data and behavior.

1.6 Key Terms

abstract data type abstraction algorithm

class computable data abstraction
data structure data type deep equality
dictionary encapsulation exception
format operator formatted strings implementation-independent
information hiding list list comprehension
method mutability object
procedural abstraction programming prompt
self shallow equality simulation
string truth table

1.7 Programming Exercises

1. Implement the simple methods get_num and get_den that will return the numerator
and denominator of a fraction.
2. In many ways it would be better if all fractions were maintained in lowest terms right
from the start. Modify the constructor for the Fraction class so that GCD is used to
reduce fractions immediately. Notice that this means the __add__ function no longer
needs to reduce. Make the necessary modifications.
3. Implement the remaining simple arithmetic operators (__sub__, __mul__, and
__truediv__).
4. Implement the remaining relational operators (__gt__, __ge__, __lt__, __le__, and
__ne__)

38 Chapter 1. Introduction
 Problem Solving with Algorithms and Data Structures, Release 3.0

5. Modify the constructor for the fraction class so that it checks to make sure that the nu-
merator and denominator are both integers. If either is not an integer the constructor
should raise an exception.
6. In the definition of fractions we assumed that negative fractions have a negative numera-
tor and a positive denominator. Using a negative denominator would cause some of the
relational operators to give incorrect results. In general, this is an unnecessary constraint.
Modify the constructor to allow the user to pass a negative denominator so that all of the
operators continue to work properly.
7. Research the __radd__ method. How does it differ from __add__? When is it used?
Implement __radd__.
8. Repeat the last question but this time consider the __iadd__ method.
9. Research the __repr__ method. How does it differ from __str__? When is it used?
Implement __repr__.
10. Design a class to represent a playing card. Now design a class to represent a deck of
cards. Using these two classes, implement a favorite card game.
11. Find a Sudoku puzzle in the local newspaper. Write a program to solve the puzzle.

1.7. Programming Exercises 39

Problem Solving with Algorithms and Data Structures, Release 3.0

40 Chapter 1. Introduction
 CHAPTER

TWO

ALGORITHM ANALYSIS

2.1 Objectives

• To understand why algorithm analysis is important.

• To be able to use “Big-O” to describe execution time.
• To understand the “Big-O” execution time of common operations on Python lists and
dictionaries.
• To understand how the implementation of Python data impacts algorithm analysis.
• To understand how to benchmark simple Python programs.

2.2 What Is Algorithm Analysis?

It is very common for beginning computer science students to compare their programs with
one another. You may also have noticed that it is common for computer programs to look very
similar, especially the simple ones. An interesting question often arises. When two programs
solve the same problem but look different, is one program better than the other?
In order to answer this question, we need to remember that there is an important difference
between a program and the underlying algorithm that the program is representing. As we stated
in Chapter 1, an algorithm is a generic, step-by-step list of instructions for solving a problem.
It is a method for solving any instance of the problem such that given a particular input, the
algorithm produces the desired result. A program, on the other hand, is an algorithm that has
been encoded into some programming language. There may be many programs for the same
algorithm, depending on the programmer and the programming language being used.
To explore this difference further, consider the function shown below. This function solves a
familiar problem, computing the sum of the first 𝑛 integers. The algorithm uses the idea of an
accumulator variable that is initialized to 0. The solution then iterates through the 𝑛 integers,
adding each to the accumulator.

def sum_of_n(n):
the_sum = 0
for i in range(1,n+1):

41
Problem Solving with Algorithms and Data Structures, Release 3.0

the_sum = the_sum + i

return the_sum

print(sum_of_n(10))

Now look at the function foo below. At first glance it may look strange, but upon further
inspection you can see that this function is essentially doing the same thing as the previous
one. The reason this is not obvious is poor coding. We did not use good identifier names to
assist with readability, and we used an extra assignment statement during the accumulation
step that was not really necessary.

def foo(tom):
fred = 0
for bill in range(1, tom+1):
barney = bill
fred = fred + barney

return fred

print(foo(10))

The question we raised earlier asked whether one function is better than another. The answer
depends on your criteria. The function sum_of_n is certainly better than the function foo if
you are concerned with readability. In fact, you have probably seen many examples of this
in your introductory programming course since one of the goals there is to help you write
programs that are easy to read and easy to understand. In this course, however, we are also
interested in characterizing the algorithm itself. (We certainly hope that you will continue to
strive to write readable, understandable code.)
Algorithm analysis is concerned with comparing algorithms based upon the amount of com-
puting resources that each algorithm uses. We want to be able to consider two algorithms and
say that one is better than the other because it is more efficient in its use of those resources
or perhaps because it simply uses fewer. From this perspective, the two functions above seem
very similar. They both use essentially the same algorithm to solve the summation problem.
At this point, it is important to think more about what we really mean by computing resources.
There are two different ways to look at this. One way is to consider the amount of space
or memory an algorithm requires to solve the problem. The amount of space required by a
problem solution is typically dictated by the problem instance itself. Every so often, however,
there are algorithms that have very specific space requirements, and in those cases we will be
very careful to explain the variations.
As an alternative to space requirements, we can analyze and compare algorithms based on
the amount of time they require to execute. This measure is sometimes referred to as the
“execution time” or “running time” of the algorithm. One way we can measure the execution
time for the function sum_of_n is to do a benchmark analysis. This means that we will track
the actual time required for the program to compute its result. In Python, we can benchmark a
function by noting the starting time and ending time with respect to the system we are using. In
the time module there is a function called time that will return the current system clock time

42 Chapter 2. Algorithm Analysis

 Problem Solving with Algorithms and Data Structures, Release 3.0

in seconds since some arbitrary starting point. By calling this function twice, at the beginning
and at the end, and then computing the difference, we can get an exact number of seconds
(fractions in most cases) for execution.

import time

def sum_of_n_2(n):
start = time.time()

the_sum = 0
for i in range(1, n+1):
the_sum = the_sum + i

end = time.time()

return the_sum,end-start

This code shows the original sum_of_n function with the timing calls embedded before and
after the summation. The function returns a tuple consisting of the result and the amount of
time (in seconds) required for the calculation. If we perform 5 invocations of the function,
each computing the sum of the first 10, 000 integers, we get the following:

>>>for i in range(5):
print("Sum is %d required %10.7f seconds" % sum_of_n_2(10000))
Sum is 50005000 required 0.0018950 seconds
Sum is 50005000 required 0.0018620 seconds
Sum is 50005000 required 0.0019171 seconds
Sum is 50005000 required 0.0019162 seconds
Sum is 50005000 required 0.0019360 seconds
>>>

Again, the time required for each run, although longer, is very consistent, averaging about 10
times more seconds. For n equal to 1, 000, 000 we get:

>>>for i in range(5):
print("Sum is %d required %10.7f seconds" %
sum_of_n_2(1000000))
Sum is 500000500000 required 0.1948988 seconds
Sum is 500000500000 required 0.1850290 seconds
Sum is 500000500000 required 0.1809771 seconds
Sum is 500000500000 required 0.1729250 seconds
Sum is 500000500000 required 0.1646299 seconds
>>>

In this case, the average again turns out to be about 10 times the previous.
Now consider the following code, which shows a different means of solving
∑︀𝑛the summation
problem. This function, sum_of_n_3, takes advantage of a closed equation 𝑖=0 𝑖 = (𝑛)(𝑛+1)
2

2.2. What Is Algorithm Analysis? 43

Problem Solving with Algorithms and Data Structures, Release 3.0

to compute the sum of the first n integers without iterating.

def sum_of_n_3(n):
return (n * (n + 1)) / 2

print(sum_of_n_3(10))

we do the same benchmark measurement for sum_of_n_3, using five different values for n
(10, 000, 100, 000, 1, 000, 000, 10, 000, 000, and 100, 000, 000), we get the following results:

Sum is 50005000 required 0.00000095 seconds

Sum is 5000050000 required 0.00000191 seconds
Sum is 500000500000 required 0.00000095 seconds
Sum is 50000005000000 required 0.00000095 seconds
Sum is 5000000050000000 required 0.00000119 seconds

There are two important things to notice about this output. First, the times recorded above are
shorter than any of the previous examples. Second, they are very consistent no matter what the
value of n. It appears that sum_of_n_3 is hardly impacted by the number of integers being
added.
But what does this benchmark really tell us? Intuitively, we can see that the iterative solutions
seem to be doing more work since some program steps are being repeated. This is likely the
reason it is taking longer. Also, the time required for the iterative solution seems to increase
as we increase the value of n. However, there is a problem. If we ran the same function on
a different computer or used a different programming language, we would likely get different
results. It could take even longer to perform sum_of_n_3 if the computer were older.
We need a better way to characterize these algorithms with respect to execution time. The
benchmark technique computes the actual time to execute. It does not really provide us with
a useful measurement, because it is dependent on a particular machine, program, time of day,
compiler, and programming language. Instead, we would like to have a characterization that
is independent of the program or computer being used. This measure would then be useful for
judging the algorithm alone and could be used to compare algorithms across implementations.

2.2.1 Big-O Notation

When trying to characterize an algorithm’s efficiency in terms of execution time,independent

of any particular program or computer, it is important to quantify the number of operations or
steps that the algorithm will require. If each of these steps is considered to be a basic unit of
computation, then the execution time for an algorithm can be expressed as the number of steps
required to solve the problem. Deciding on an appropriate basic unit of computation can be a
complicated problem and will depend on how the algorithm is implemented.
A good basic unit of computation for comparing the summation algorithms shown earlier might
be to count the number of assignment statements performed to compute the sum. In the function
sum_of_n, the number of assignment statements is 1 (the_sum= 0) plus the value of n (the
number of times we perform the_sum=the_sum+𝑖). We can denote this by a function, call it

44 Chapter 2. Algorithm Analysis

 Problem Solving with Algorithms and Data Structures, Release 3.0

𝑇 , where 𝑇 (𝑛) = 1 + 𝑛. The parameter 𝑛 is often referred to as the “size of the problem,” and
we can read this as “𝑇 (𝑛) is the time it takes to solve a problem of size 𝑛, namely 1 + 𝑛 steps.”
In the summation functions given above, it makes sense to use the number of terms in the
summation to denote the size of the problem. We can then say that the sum of the first 100, 000
integers is a bigger instance of the summation problem than the sum of the first 1, 000. Because
of this, it might seem reasonable that the time required to solve the larger case would be greater
than for the smaller case. Our goal then is to show how the algorithm’s execution time changes
with respect to the size of the problem.
Computer scientists prefer to take this analysis technique one step further. It turns out that the
exact number of operations is not as important as determining the most dominant part of the
𝑇 (𝑛) function. In other words, as the problem gets larger, some portion of the 𝑇 (𝑛) function
tends to overpower the rest. This dominant term is what, in the end, is used for comparison. The
order of magnitude function describes the part of 𝑇 (𝑛) that increases the fastest as the value
of 𝑛 increases. Order of magnitude is often called Big-O notation (for “order”) and written as
𝑂(𝑓 (𝑛)). It provides a useful approximation to the actual number of steps in the computation.
The function 𝑓 (𝑛) provides a simple representation of the dominant part of the original 𝑇 (𝑛).
In the above example, 𝑇 (𝑛) = 1 + 𝑛. As 𝑛 gets large, the constant 1 will become less and
less significant to the final result. If we are looking for an approximation for 𝑇 (𝑛), then we
can drop the 1 and simply say that the running time is 𝑂(𝑛). It is important to note that the 1
is certainly significant for 𝑇 (𝑛). However, as 𝑛 gets large, our approximation will be just as
accurate without it.
As another example, suppose that for some algorithm, the exact number of steps is 𝑇 (𝑛) =
5𝑛2 + 27𝑛 + 1005. When 𝑛 is small, say 1 or 2, the constant 1005 seems to be the dominant
part of the function. However, as 𝑛 gets larger, the 𝑛2 term becomes the most important. In
fact, when 𝑛 is really large, the other two terms become insignificant in the role that they play
in determining the final result. Again, to approximate 𝑇 (𝑛) as 𝑛 gets large, we can ignore the
other terms and focus on 5𝑛2 . In addition, the coefficient 5 becomes insignificant as 𝑛 gets
large. We would say then that the function 𝑇 (𝑛) has an order of magnitude 𝑓 (𝑛) = 𝑛2 , or
simply that it is 𝑂(𝑛2 ).
Although we do not see this in the summation example, sometimes the performance of an
algorithm depends on the exact values of the data rather than simply the size of the problem.
For these kinds of algorithms we need to characterize their performance in terms of best case,
worst case, or average case performance. The worst case performance refers to a particular
data set where the algorithm performs especially poorly. Whereas a different data set for the
exact same algorithm might have extraordinarily good performance. However, in most cases the
algorithm performs somewhere in between these two extremes (average case). It is important
for a computer scientist to understand these distinctions so they are not misled by one particular
case.
A number of very common order of magnitude functions will come up over and over as you
study algorithms. These are shown in Table 2.1. In order to decide which of these functions is
the dominant part of any 𝑇 (𝑛) function, we must see how they compare with one another as 𝑛
gets large.

Figure 2.1 shows graphs of the common functions from Table 2.1. Notice that when 𝑛 is small,

2.2. What Is Algorithm Analysis? 45

Problem Solving with Algorithms and Data Structures, Release 3.0

f(n) Name
1 Constant
log 𝑛 Logarithmic
𝑛 Linear
𝑛 log 𝑛 Log Linear
𝑛2 Quadratic
𝑛3 Cubic
2𝑛 Exponential

Table 2.1: Common Functions for Big-O

Figure 2.1: Plot of Common Big-O Functions

the functions are not very well defined with respect to one another. It is hard to tell which is
dominant. However, as 𝑛 grows, there is a definite relationship and it is easy to see how they
compare with one another.

As a final example, suppose that we have the fragment of Python code shown below. Although
this program does not really do anything, it is instructive to see how we can take actual code
and analyze performance.

a = 5
b = 6
c = 10
for i in range(n):
for j in range(n):
x = i * i
y = j * j

46 Chapter 2. Algorithm Analysis

 Another Random Document on
Scribd Without Any Related Topics
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com