Pro Python System Administration 2nd Edition Rytis Sileika download
Pro Python System Administration 2nd Edition Rytis Sileika download
https://ebookfinal.com/download/pro-python-system-
administration-2nd-edition-rytis-sileika/
https://ebookfinal.com/download/automating-linux-and-unix-system-
administration-2nd-edition-nathan-campi/
https://ebookfinal.com/download/principles-of-network-and-system-
administration-2nd-ed-edition-mark-burgess/
https://ebookfinal.com/download/mastering-linux-system-
administration-1st-edition-christine-bresnahan/
https://ebookfinal.com/download/unix-and-linux-system-administration-
handbook-fourth-edition-evi-nemeth/
LDAP System Administration Putting Directories to Work 1st
Edition Carter
https://ebookfinal.com/download/ldap-system-administration-putting-
directories-to-work-1st-edition-carter/
https://ebookfinal.com/download/pro-exchange-administration-
understanding-on-premises-and-hybrid-exchange-deployments-third-
edition-jaap-wesselius/
https://ebookfinal.com/download/automating-system-administration-with-
perl-tools-to-make-you-more-efficient-2nd-ed-edition-david-n-blank-
edelman/
https://ebookfinal.com/download/pro-puppet-2nd-edition-spencer-krum/
https://ebookfinal.com/download/python-in-easy-steps-covers-
python-3-7-2nd-edition-mike-mcgrath/
Pro Python System Administration 2nd Edition Rytis
Sileika Digital Instant Download
Author(s): Rytis Sileika
ISBN(s): 9781484202180, 148420218X
File Details: PDF, 5.48 MB
Year: 2014
Language: english
www.it-ebooks.info
For your convenience Apress has placed some of the front
matter material after the index. Please use the Bookmarks
and Contents at a Glance links to access them.
www.it-ebooks.info
Contents at a Glance
Index���������������������������������������������������������������������������������������������������������������������������������391
www.it-ebooks.info
Introduction
The role of the system administrator has grown dramatically over the years. The number of systems supported
by a single engineer has also increased. As such, it is impractical to handcraft each installation, and there is a
need to automate as many tasks as possible. The structure of systems varies from organization to organization,
therefore system administrators must be able to create their own management tools. Historically, the most popular
programming languages for these tasks were UNIX shell and Perl. They served their purposes well, and I doubt they
will ever cease to exist. However, the complexity of current systems requires new tools, and the Python programming
language is one of them.
Python is an object-oriented programming language suitable for developing large-scale applications. Its syntax
and structure make it very easy to read—so much so that the language is sometimes referred to as “executable
pseudocode.” The Python interpreter allows for interactive execution, so in some situations an administrator can use
it instead of a standard UNIX shell. Although Python is primarily an object-oriented language, it is easily adopted
for procedural and functional styles of programming. Given all that, Python makes a perfect fit as a new language for
implementing system administration applications. There are a large number of Linux system utilities already written
in Python, such as the Yum package manager and Anaconda, the Linux installation program.
xxiii
www.it-ebooks.info
■ Introduction
xxiv
www.it-ebooks.info
■ Introduction
Chapter 6: Gathering and Presenting Statistical Data from Apache Log Files
In this chapter, the goal is to build an application that parses and analyses the Apache web server log files.
Instead of taking the straightforward but inflexible approach of building a monolithic application, we look at the
design principles involved in building plug-in applications. You learn how to use the object and class type discovery
functions and how to perform a dynamic module loading. This second edition of the book shows you how to perform
data visualization based on the gathered data.
Chapter 7: Performing Complex Searches and Reporting on Application Log Files
This chapter also deals with the log file parsing, but this time I show you how to parse complex, multi-line log
file entries. We investigate the functionality of the open-source log file parser tool called Exctractor, which you can
download from http://exctractor.sourceforge.net/.
Chapter 8: A Web Site Availability Check Script for Nagios
Nagios is one of the most popular open-source monitoring systems, because its modular structure allows users
to implement their own check scripts and thus customize the tool to meet their needs. In this chapter, we create two
scripts that check the functionality of a website. We investigate how to use the Beautiful Soup HTML parsing library to
extract the information from the HTML web pages.
Chapter 9: Management and Monitoring Subsystem
This chapter starts a three-chapter series in which we build a complete monitoring system. The goal of this
chapter is not to replace mature monitoring systems such as Nagios or Zenoss but to show the basic principles of the
distributed application programming. We look at database design principles such as data normalization. We also
investigate how to implement the communication mechanisms between network services using the RPC calls.
Chapter 10: Remote Monitoring Agents
This is the second chapter in the monitoring series, where we implement the remote monitoring agent
components. In this chapter, I also describe how to decouple the application from its configuration using the
ConfigParser module.
Chapter 11: Statistics Gathering and Reporting
This is the last part of the monitoring series, where I show you how to perform basic statistical analysis on the
collected performance data. We use scientific libraries: NumPy to perform the calculations and matplotlib to create
the graphs. You learn how to find which performance readings fall into the comfort zone and how to calculate the
boundaries of that zone. We also do the basic trend detection, which provides a good insight for the capacity planning.
Chapter 12: Distributed Message Processing System
This is a new chapter for the second edition of the book. In this chapter I show you how to convert the distributed
management system to use Celery, a remote task execution framework.
Chapter 13: Automatic MySQL Database Performance Tuning
In this chapter, I show you how to obtain the MySQL database configuration variables and the internal status
indicators. We build an application that makes a suggestion on how to improve the database engine performance
based on the obtained data.
Chapter 14: Amazon EC2/S3 as a Data Warehouse Solution
This chapter shows you how to utilize the Amazon Elastic Compute Cloud (EC2) and offload the infrequent
computation tasks to it. We build an application that automatically creates a database server where you can transfer
data for further analysis. You can use this example as a basis to build an on-demand data warehouse solution.
xxv
www.it-ebooks.info
■ Introduction
xxvi
www.it-ebooks.info
Chapter 1
Most devices that are connected to a network report their status using SNMP (the Simple Network Management
Protocol). This protocol was designed primarily for managing and monitoring network-attached hardware devices,
but some applications also expose their statistical data using this protocol. In this chapter we will look at how to
access this information from your Python applications. We are going to store the obtained data in an RRD (round robin
database), using RRDTool—a widely known and popular application and library, which is used to store and plot the
performance data. Finally we’ll investigate the Jinja2 template system, which we’ll use to generate simple web pages
for our application.
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
Finally, we need to generate the graphs and represent this information on easily accessible web pages. The
information needs to be structured by the device names only. For example, if we are monitoring several devices for CPU
and network interface utilization, this information needs to be presented on a single page. We don’t need to present this
information on multiple time scales; by default the graphs should show the performance indicators for the last 24 hours.
Introduction to SNMP
SNMP (Simple Network Management Protocol) is a UDP-based protocol used mostly for managing network-attached
devices, such as routers, switches, computers, printers, video cameras, and so on. Some applications also allow access
to internal counters via the SNMP protocol.
SNMP not only allows you to read performance statistics from the devices, it can also send control messages to
instruct a device to perform some action—for example, you can restart a router remotely by using SNMP commands.
There are three main components in a system managed by SIMPLE NETWORK MANAGEMENT
PROTOCOL (SNMP):
• The management system which is responsible for managing all devices
• The managed devices, which are all devices managed by the management system
• The SNMP agent, which is an application that runs on each of the managed devices and
interacts with the management system
This relationship is illustrated in Figure 1-1.
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
The Management
System
This approach is rather generic. The protocol defines seven basic commands, of which the most interesting to us
are get, get bulk, and response. As you may have guessed, the former two are the commands that the management
system issues to the agent, and the latter is a response from the agent software.
How does the management system know what to look for? The protocol does not define a way of exchanging
this information, and therefore the management system has no way to interrogate the agents to obtain the list of
available variables.
The issue is resolved by using a Management Information Base (or MIB). Each device usually has an associated
MIB, which describes the structure of the management data on that system. Such a MIB would list in hierarchical
order all object identifiers (OIDs) that are available on the managed device. The OID effectively represents a node
in the object tree. It contains numerical identifiers of all nodes leading to the current OID starting from the node at
the top of the tree. The node IDs are assigned and regulated by the IANA (Internet Assigned Numbers Authority).
An organization can apply for an OID node, and when it is assigned it is responsible for managing the OID structure
below the allocated node.
Figure 1-2 illustrates a portion of the OID tree.
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
ROOT
ORG (3)
DOD (6)
INTERNET (1)
System (1)
Interfaces (2)
sysDescr (1) sysUpTime (3)
ifEntry (1)
Let’s look at some example OIDs. The OID tree node that is assigned to the Cisco organization has a value of
1.3.6.1.4.1.9, which means that all proprietary OIDs that are associated with the Cisco manufactured devices will start
with these numbers. Similarly, the Novell devices will have their OIDs starting with 1.3.6.1.4.1.23.
I deliberately emphasized proprietary OIDs because some properties are expected to be present (if and where
available) on all devices. These are under the 1.3.6.1.2.1.1 (System SNMP Variables) node, which is defined by
RFC1213. For more details on the OID tree and its elements, visit http://www.alvestrand.no/objectid/top.html.
This website allows you to browse the OID tree and it contains quite a large collection of the various OIDs.
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
(continued)
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
Authentication in SNMP
Authentication in earlier SNMP implementations is somewhat primitive and is prone to attacks. An SNMP agent
defines two community strings: one for read-only access and the other for read/write access. When the management
system connects to the agent, it must authenticate with one of those two strings. The agent accepts commands only
from a management system that has authenticated with valid community strings.
The most useful command from this package is snmpwalk, which takes an OID node as an argument and tries
to discover all subnode OIDs. This command uses the SNMP operation getnext, which returns the next node in the
tree and effectively allows you to traverse the whole subtree from the indicated node. If no OID has been specified,
snmpwalk will use the default SNMP system OID (1.3.6.1.2.1) as the starting point. Listing 1-1 demonstrates the
snmpwalk command issued against a laptop running Fedora Linux.
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
.1.3.6.1.2.1.2.2.1.14.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.14.3 = Counter32: 0
.1.3.6.1.2.1.2.2.1.14.4 = Counter32: 0
.1.3.6.1.2.1.2.2.1.14.5 = Counter32: 0
.1.3.6.1.2.1.2.2.1.15.1 = Counter32: 0
.1.3.6.1.2.1.2.2.1.15.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.15.3 = Counter32: 0
.1.3.6.1.2.1.2.2.1.15.4 = Counter32: 0
.1.3.6.1.2.1.2.2.1.15.5 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.1 = Counter32: 89275
.1.3.6.1.2.1.2.2.1.16.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.3 = Counter32: 922277
.1.3.6.1.2.1.2.2.1.16.4 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.5 = Counter32: 3648
.1.3.6.1.2.1.2.2.1.17.1 = Counter32: 1092
.1.3.6.1.2.1.2.2.1.17.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.17.3 = Counter32: 7540
.1.3.6.1.2.1.2.2.1.17.4 = Counter32: 0
.1.3.6.1.2.1.2.2.1.17.5 = Counter32: 17
.1.3.6.1.2.1.2.2.1.18.1 = Counter32: 0
.1.3.6.1.2.1.2.2.1.18.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.18.3 = Counter32: 0
.1.3.6.1.2.1.2.2.1.18.4 = Counter32: 0
.1.3.6.1.2.1.2.2.1.18.5 = Counter32: 0
.1.3.6.1.2.1.2.2.1.19.1 = Counter32: 0
.1.3.6.1.2.1.2.2.1.19.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.19.3 = Counter32: 0
.1.3.6.1.2.1.2.2.1.19.4 = Counter32: 0
.1.3.6.1.2.1.2.2.1.19.5 = Counter32: 0
.1.3.6.1.2.1.2.2.1.20.1 = Counter32: 0
.1.3.6.1.2.1.2.2.1.20.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.20.3 = Counter32: 0
.1.3.6.1.2.1.2.2.1.20.4 = Counter32: 0
.1.3.6.1.2.1.2.2.1.20.5 = Counter32: 0
.1.3.6.1.2.1.2.2.1.21.1 = Gauge32: 0
.1.3.6.1.2.1.2.2.1.21.2 = Gauge32: 0
.1.3.6.1.2.1.2.2.1.21.3 = Gauge32: 0
.1.3.6.1.2.1.2.2.1.21.4 = Gauge32: 0
.1.3.6.1.2.1.2.2.1.21.5 = Gauge32: 0
.1.3.6.1.2.1.2.2.1.22.1 = OID: .0.0
.1.3.6.1.2.1.2.2.1.22.2 = OID: .0.0
.1.3.6.1.2.1.2.2.1.22.3 = OID: .0.0
.1.3.6.1.2.1.2.2.1.22.4 = OID: .0.0
.1.3.6.1.2.1.2.2.1.22.5 = OID: .0.0
.1.3.6.1.2.1.25.1.1.0 = Timeticks: (8232423) 22:52:04.23
.1.3.6.1.2.1.25.1.1.0 = No more variables left in this MIB View (It is past the end
of the MIB tree)
As an exercise, try to identify some of the listed OIDs using Tables 1-1 and 1-2 and find out what they mean.
10
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
[check_1]
description=WLAN incoming traffic
oid=1.3.6.1.2.1.2.2.1.10.3
system=system_1
[check_2]
description=WLAN incoming traffic
oid=1.3.6.1.2.1.2.2.1.16.3
system=system_1
Make sure that the system and check section IDs are unique, or you may get unpredictable results.
We’re going to create an SnmpManager class with two methods, one to add a system and the other to add a check.
As the check contains the system ID string, it will automatically be assigned to that particular system. In Listing 1-3
you can see the class definition and also the initialization part that reads in the configuration and iterates through the
sections and updates the class object accordingly. Create a file called snmp-manage.py with the contents shown in the
listing below. We will work on adding new features to the script as we go along.
11
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
def main(conf_file=""):
if not conf_file:
sys.exit(-1)
config = SafeConfigParser()
config.read(conf_file)
snmp_manager = SnmpManager()
for system in [s for s in config.sections() if s.startswith('system')]:
snmp_manager.add_system(system,
config.get(system, 'description'),
config.get(system, 'address'),
config.get(system, 'port'),
config.get(system, 'communityro'))
for check in [c for c in config.sections() if c.startswith('check')]:
snmp_manager.add_check(check,
config.get(check, 'oid'),
config.get(check, 'description'),
config.get(check, 'system'))
if __name__ == '__main__':
main(conf_file='snmp-manager.cfg')
As you see in the example, we first have to iterate through the system sections and update the object before
proceeding with the check sections.
■■Note This order is important, because if we try to add a check for a system that hasn’t been inserted yet, we’ll get a
dictionary index error.
12
www.it-ebooks.info
Chapter 1 ■ Reading and Collecting Performance Data Using SNMP
Also note that we are converting the OID string to a tuple of integers. You’ll see why we have to do this later in this
section. The configuration file is loaded and we’re ready to run SNMP queries against the configured devices.
Alternatively, you can use the Python Package manager (PiP) to install this library for you:
If you don’t have the pip command available, you can download and install this tool from
http://pypi.python.org/pypi/pip. We will use it in later chapters as well.
The PySNMP library hides all the complexity of SNMP processing behind a single class with a simple API.
All you have to do is create an instance of the CommandGenerator class. This class is available from the
pysnmp.entity.rfc3413.oneliner.cmdgen module and implements most of the standard SNMP protocol
commands: getCmd(), setCmd(), and nextCmd(). Let’s look at each of these in more detail.
13
www.it-ebooks.info
Discovering Diverse Content Through
Random Scribd Documents
The Project Gutenberg eBook of Christmas offering
This ebook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this ebook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.
Language: English
CHRISTMAS OFFERING:
BY MRS. ELIZABETH DIMOND,
OF BRISTOL, R. I.
PROVIDENCE:
PRINTED BY KNOWLES AND VOSE.
1847.
MRS. SMITH,
Providence, R. I.
Dear Friend:
I am yours, &c.
ELIZABETH DIMOND.
FOOTNOTES:
[A] After a great fall of snow, sleds, with cattle attached, filled with men and boys,
with floating banners and merry bells, cut their way through the snow; thus making
the streets passable for lighter vehicles, and for foot passengers.
[B] A name applied by a friend, denoting freshness and vivacity. The Gillyflower
united denotes the sweetness of her temper.
[C] “O’er the ferry way,” the residence of my friend’s brother, a delightful farm
situated between Bristol and Newport, called the “Elam Place.”
*** END OF THE PROJECT GUTENBERG EBOOK CHRISTMAS
OFFERING ***
1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside
the United States, check the laws of your country in addition to
the terms of this agreement before downloading, copying,
displaying, performing, distributing or creating derivative works
based on this work or any other Project Gutenberg™ work. The
Foundation makes no representations concerning the copyright
status of any work in any country other than the United States.
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookfinal.com