Python Data Analysis Second Edition Armando Fandangoinstant download
Python Data Analysis Second Edition Armando Fandangoinstant download
https://ebookmeta.com/product/python-data-analysis-second-
edition-armando-fandango/
https://ebookmeta.com/product/python-for-data-analysis-3rd-
edition-second-early-release-wes-mckinney/
https://ebookmeta.com/product/python-data-analysis-numpy-
matplotlib-and-pandas-bernd-klein/
https://ebookmeta.com/product/clojure-data-analysis-cookbook-
second-edition-rochester-eric/
https://ebookmeta.com/product/their-little-pickpocket-1st-
edition-aidyn-rose-2/
Introduction to Scilab: For Engineers and Scientists
1st Edition Sandeep Nagar
https://ebookmeta.com/product/introduction-to-scilab-for-
engineers-and-scientists-1st-edition-sandeep-nagar/
https://ebookmeta.com/product/introducing-japanese-popular-
culture-2nd-edition-alisa-freedman/
https://ebookmeta.com/product/s-more-love-fall-in-love-forever-
safe-book-6-1st-edition-jaime-russell/
https://ebookmeta.com/product/a-monster-for-every-season-
spring-2-1st-edition-rich-burlew/
https://ebookmeta.com/product/engineering-deep-learning-systems-
meap-v04-1-meap-v04-edition-chi-wang/
Heir of Fire Sarah J Maas
https://ebookmeta.com/product/heir-of-fire-sarah-j-maas/
Table of Contents
Python Data Analysis - Second Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Why subscribe?
Customer Feedback
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
1. Getting Started with Python Libraries
Installing Python 3
Installing data analysis libraries
On Linux or Mac OS X
On Windows
Using IPython as a shell
Reading manual pages
Jupyter Notebook
NumPy arrays
A simple application
Where to find help and references
Listing modules inside the Python libraries
Visualizing data using Matplotlib
Summary
2. NumPy Arrays
The NumPy array object
Advantages of NumPy arrays
Creating a multidimensional array
Selecting NumPy array elements
NumPy numerical types
Data type objects
Character codes
The dtype constructors
The dtype attributes
One-dimensional slicing and indexing
Manipulating array shapes
Stacking arrays
Splitting NumPy arrays
NumPy array attributes
Converting arrays
Creating array views and copies
Fancy indexing
Indexing with a list of locations
Indexing NumPy arrays with Booleans
Broadcasting NumPy arrays
Summary
References
3. The Pandas Primer
Installing and exploring Pandas
The Pandas DataFrames
The Pandas Series
Querying data in Pandas
Statistics with Pandas DataFrames
Data aggregation with Pandas DataFrames
Concatenating and appending DataFrames
Joining DataFrames
Handling missing values
Dealing with dates
Pivot tables
Summary
References
4. Statistics and Linear Algebra
Basic descriptive statistics with NumPy
Linear algebra with NumPy
Inverting matrices with NumPy
Solving linear systems with NumPy
Finding eigenvalues and eigenvectors with NumPy
NumPy random numbers
Gambling with the binomial distribution
Sampling the normal distribution
Performing a normality test with SciPy
Creating a NumPy masked array
Disregarding negative and extreme values
Summary
5. Retrieving, Processing, and Storing Data
Writing CSV files with NumPy and Pandas
The binary .npy and pickle formats
Storing data with PyTables
Reading and writing Pandas DataFrames to HDF5 stores
Reading and writing to Excel with Pandas
Using REST web services and JSON
Reading and writing JSON with Pandas
Parsing RSS and Atom feeds
Parsing HTML with Beautiful Soup
Summary
Reference
6. Data Visualization
The matplotlib subpackages
Basic matplotlib plots
Logarithmic plots
Scatter plots
Legends and annotations
Three-dimensional plots
Plotting in Pandas
Lag plots
Autocorrelation plots
Plot.ly
Summary
7. Signal Processing and Time Series
The statsmodels modules
Moving averages
Window functions
Defining cointegration
Autocorrelation
Autoregressive models
ARMA models
Generating periodic signals
Fourier analysis
Spectral analysis
Filtering
Summary
8. Working with Databases
Lightweight access with sqlite3
Accessing databases from Pandas
SQLAlchemy
Installing and setting up SQLAlchemy
Populating a database with SQLAlchemy
Querying the database with SQLAlchemy
Pony ORM
Dataset - databases for lazy people
PyMongo and MongoDB
Storing data in Redis
Storing data in memcache
Apache Cassandra
Summary
9. Analyzing Textual Data and Social Media
Installing NLTK
About NLTK
Filtering out stopwords, names, and numbers
The bag-of-words model
Analyzing word frequencies
Naive Bayes classification
Sentiment analysis
Creating word clouds
Social network analysis
Summary
10. Predictive Analytics and Machine Learning
Preprocessing
Classification with logistic regression
Classification with support vector machines
Regression with ElasticNetCV
Support vector regression
Clustering with affinity propagation
Mean shift
Genetic algorithms
Neural networks
Decision trees
Summary
11. Environments Outside the Python Ecosystem and Cloud
Computing
Exchanging information with Matlab/Octave
Installing rpy2 package
Interfacing with R
Sending NumPy arrays to Java
Integrating SWIG and NumPy
Integrating Boost and Python
Using Fortran code through f2py
PythonAnywhere Cloud
Summary
12. Performance Tuning, Profiling, and Concurrency
Profiling the code
Installing Cython
Calling C code
Creating a process pool with multiprocessing
Speeding up embarrassingly parallel for loops with Joblib
Comparing Bottleneck to NumPy functions
Performing MapReduce with Jug
Installing MPI for Python
IPython Parallel
Summary
A. Key Concepts
B. Useful Functions
Matplotlib
NumPy
Pandas
Scikit-learn
SciPy
scipy.fftpack
scipy.signal
scipy.stats
C. Online Resources
Python Data Analysis -
Second Edition
Python Data Analysis -
Second Edition
Copyright © 2017 Packt Publishing
Every effort has been made in the preparation of this book to ensure
the accuracy of the information presented. However, the information
contained in this book is sold without warranty, either express or
implied. Neither the authors, nor Packt Publishing, and its dealers
and distributors will be held liable for any damages caused or
alleged to be caused directly or indirectly by this book.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78712-748-7
www.packtpub.com
Credits
Reviewers
Project Coordinator
Joran Beasley
Shweta H Birwatkar
Ratan Kumar
Armando has worked for more than ten years in projects involving
predictive analytics, data science, machine learning, big data,
product engineering, high performance computing, and cloud
infrastructures. His research interests spans machine learning, deep
learning, and scientific computing.
Did you know that Packt offers eBook versions of every book
published, with PDF and ePub files available? You can upgrade to the
eBook version at www.PacktPub.com and as a print book customer,
you are entitled to a discount on the eBook copy. Get in touch with
us at service@packtpub.com for more details.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you
full access to all Packt books and video courses, as well as industry-
leading tools to help you plan your personal development and
advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Discovering Diverse Content Through
Random Scribd Documents
condone his great crime. Queen Victoria, the typical British matron,
exchanging visits with the Imperial adventurer made an edifying
spectacle! Presently the land-greed of England and the gloire-thirst
of France brought the sons of the Britons who had whipped the
great Napoleon at Waterloo into an alliance with the sons of the
Frenchmen who had there been whipped; and in the summer of
1854 British and French fleets swept through the Bosphorus and
across the Black Sea, and landed two armies near Sebastopol. Of the
Crimean war which ensued, we need say no more than that it was
immoral in conception, blundering in execution, and ineffectual in
results. Nevertheless, it supplied Napoleon III with just what he had
sought. He extracted from it large quantities of gloire. Marshal’s
bâtons and military promotions, the parade of returning troops, the
assembling at Paris of the European envoys who were to agree on a
treaty of peace,—what did all this show but that Europe had
accepted Napoleon III at his own valuation? In Russia’s wilderness
of snow the great Napoleon had been ruined; now his nephew
posed as the humbler of Russia. The great Napoleon had been
finally crushed by England: now his nephew had enticed good, pious
England into an alliance, and thereby he had surely avenged his
uncle. The last European compact, humiliating to France, had been
signed at Vienna: the new compact, signed at Paris, bore witness to
the supremacy of France.
That year 1856 marks the acme of Napoleon the Third’s career. It
saw him the recognized arbiter of Europe. The world, which
worships success, forgot that the suave, impassive master of the
Tuileries had been Louis the Ridiculed, a political vagabond and
hapless pretender, only ten years before. Now, as arbiter, he would
meddle when he chose, and the world should not gainsay him.
Moreover, he believed his power so secure that he was willing to
forgive those whom he had injured. He had gained what he wanted:
why, therefore, should they reject his amnesty?
Just as the Luxemburg affair was concluded, all the world went to
Paris to attend the Exposition, which was intended to be, and
seemed, a symbol of the permanence of the Second Empire. The
projectors knew that the immense preparations would enable the
government to employ many workmen, who might otherwise be
unruly, and that the vast concourse of visitors would bring money to
the tradespeople and keep them from grumbling. The ostensible
purpose, however, was to dazzle both Frenchmen and strangers by a
view of Imperial magnificence; and it was fully achieved.
Paris herself, the Phryne among cities, astonished those who had
never seen her, or who had seen her in old days. Where, they asked,
were the narrow, crooked streets, in which barricaders once fortified
themselves? Were these boulevards, stretching broad and straight,—
were these they? And by what magic had the old, irregular dwellings
been transformed into miles of tall, stately blocks? New churches,
new quays, new parks, new palaces, bearing the impress of grace,
symmetry, and a unifying planner, excited the wonder of the
cosmopolitan throng of visitors. But the products of industry, the
triumphs of the arts of peace, were not allowed to obscure the
military glories of the Second Empire. A “Bridge of the Alma” and a
“Boulevard of Sebastopol” kept the Crimean prowess in memory; a
“Solferino Bridge” and a “Magenta Boulevard” bore witness to the
Italian triumphs. And there were pageants, military, courtly, artistic;
balls, at which, among the picked beauties of the world, the Empress
Eugénie shone most beautiful; banquets, at which Napoleon sat at
the head of the table, with monarchs at his right hand and his left
deferentially listening. Little did the on-lookers suppose that the
master of those magnificent revels had been lately frustrated by M.
de Bismarck, who was merely one of the million whose presence in
Paris seemed a tribute to Napoleon’s supremacy.
In closing, let us read, from a letter Bismarck wrote to his wife the
day after the surrender, a description of the meeting of Napoleon
and his conqueror:—
That morning the terms of capitulation were drawn up, and the
next day Napoleon went a prisoner to Wilhelmshöhe, whence, in due
time, he was allowed to depart for England. At Chislehurst, on
January 9, 1873, he died, having lived to see not only the extinction
of French Imperialism and of the temporal Papacy, but also the
creation of the German Empire and the union of Italy. To prevent all
of these things had been his aim.
The most significant event of the Diet of 1825 was the use by
Count Stephen Széchenyi of the Magyar language instead of the
Latin. Széchenyi, having traveled in Western Europe, came back
imbued with large schemes of progress. He helped to introduce
steamboats on the Danube; he founded a Magyar Academy; he
proposed to join Buda and Pesth by a suspension bridge. By
stimulating the material welfare of his country, he hoped that many
of the social abuses would vanish without a struggle. And now his
use of the Magyar language was a symptom of the awakening of the
spirit of nationality,—one of the controlling motives in the history of
Europe during the nineteenth century. In Hungary, as elsewhere, the
arousing of that spirit was evidenced not only by an intenser political
life, but also by a literary revival.
As late as 1840 both the magnates and the lesser nobility clung to
their national costume as loyally as to their national constitution. “It
now consists of the attilla” writes Paget at that date, “a frock coat,
reaching nearly to the knee, with a military collar, and covered in
front with gold lace; over this is generally worn, hanging loosely on
one shoulder, the mente, a somewhat larger coat, lined with fur, and
with a fur cape. It is generally suspended by some massive jeweled
chain. The tight pantaloons and ankle-boots, with the never-failing
spurs, form the lower part. The kalpak, or fur cap, is of innumerable
forms, and ornamented by a feather fastened by a rich brooch. The
white heron’s plume, or aigrette, the rare product of the Southern
Danube, is the most esteemed. The neck is opened, except for a
black ribbon loosely passed round it, the ends of which are finished
with gold fringe. The sabre is in the shape of the Turkish scimitar;
indeed, richly ornamented Damascus blades, the spoils of some
unsuccessful Moslem invasion, are very often worn, and are highly
prized.
Such, in its dress, was the Magyar aristocracy which the reformers
set themselves to overcome; and in their character those Magyar
nobles—were they magnates or simply gentlemen—cherished a
tenacity of class unsurpassed by any other aristocrats in Europe.
Nevertheless, the reformers boldly put forth a programme which
involved the complete social and political reorganization of the
country,—even throwing down a challenge to the aristocracy to
surrender privileges in which these deemed their very existence
rooted. Parties had begun to array themselves on these lines when
Louis Kossuth entered public life.
Doubt not that Kossuth knew how to kindle the fuel which ages of
hatred had been storing. He had the gift peculiar to really great
popular leaders of appealing directly to racial pride and passion; so it
mattered little that he dealt in generalizations. Speaking broadly, he
preached the abolition of feudalism and the aggrandizement of the
Magyar nationality. The former purpose brought him and the Liberals
into conflict with the conservative aristocracy; the latter inflamed
against the Magyars the long-smouldering hatred of their subject
peoples.
For the spirit of nationality had awakened these also. The Slavs of
Croatia, Slavonia, and Dalmatia dreamed of establishing a great
Slavic kingdom in Southeastern Europe; they, too, were putting forth
a literature. Their Illyrism—to their prospective nation they gave the
name “Illyria”—clashed with the recrudescent Magyarism. When the
Hungarian Diet decreed that the Magyar language should be taught
in their schools, and that every official must use it, they protested as
strenuously as the Magyars themselves had protested when Austria
tried to impose the German language and German officials on them.
“The Magyars are an island in the Slavic ocean,” exclaimed Gaj, the
poet and spokesman of Illyrism, to the Hungarian Diet: “I did not
make this ocean, I did not stir up its waves; but take care that they
do not go over your heads and drown you.” Nevertheless, the law
was passed. In the Southland the Serbs along the Danube, in the
East the Wallachs of Transylvania, feeling the first tingle of national
aspirations, resented this encroachment. Austria—whose motto was,
Divide et impera—found her advantage in embittering tribe with
tribe and class with class.
Thus through all the arteries of the body politic new blood was
throbbing. Give a people a great idea, and they will find how to
apply it to every concern of life. The Magyar Liberals were surely
undermining feudalism; their race was growing more and more
restive at Austria’s obstinate delays. When Austria removed the
native county sheriffs and put German administrators in their stead,
all the Magyar factions joined in denouncing such an assault on their
national life. The county system had been the safeguard of
Hungary’s political institutions for well-nigh eight hundred years; the
sheriff was the foremost official in the county, to whose guidance its
interests and civic activity were intrusted. To make an alien sheriff
was therefore to check national agitation at its source. Accordingly,
the Diet which met in the autumn of 1847 met full of defiance and
resentment, though the platform of the Liberals, drawn up by the
judicial Deák, wore on its surface a conciliatory aspect. After a hot
canvass, Kossuth was elected to represent Pesth in the Chamber of
Deputies. A few sessions sufficed to establish his preëminence as an
orator, and his leadership of the Liberal party.
During the winter months of 1847–48 but little was done, though
much was discussed. As usual, the Magnates resisted the reforms
aimed at their class; as usual, Government temporized and
postponed. Suddenly, at the beginning of March, 1848, news
reached Presburg of the revolution in Paris, and of the flight of Louis
Philippe. That news passed like a torch throughout Europe, kindling
as it passed the fires of revolt. At Presburg, on March 3, Kossuth
rose in the Diet and interrupted a debate on the financial difficulties
with Austria. That question of finance, he said, could never be
settled separately; in it was involved the whole question of Austria’s
disregard of Hungary’s rights. Hungary must have her own laws, her
own ministry; taxation must be equal; the franchise must be
extended. More than that, he added, Hungary could never prosper
until every part of the Empire should be governed by uniform
constitutional methods.
All this was not yet clear to the Hungarians. Assuming the Imperial
assurances to be honest, they passed a reform bill abolishing the
privileges of the nobles, who were to be compensated by the state
for the loss they sustained in the emancipation of their serfs. Bills
authorizing equal taxation, trial by jury, freedom of speech, the
abolition of tithes, and the extension of the franchise to one million
two hundred thousand voters, were adopted with but little
discussion. Religious toleration—except for Jews—became the law of
the land.
The Magnates having made this unparalleled sacrifice, King
Ferdinand came over to Presburg and dissolved the Diet in a speech
approving its action, and reiterating his pledge to uphold the
Constitution. The Cabinet proceeded to organize its administration,—
a task which would have been sufficient at any time to keep it busy,
but now extraordinary and urgent matters pressed upon it. The
Wallachs, Serbs, and Croats rose in rebellion. Most alarming was the
situation in Croatia, where the Slavs were agitating for separation
from Hungary. Baron Jellachich, who had just been appointed Ban or
Viceroy of Croatia, abetting the insurrection, strengthened the Croat
army. In June the Magyar ministers hurried to Innspruck—whither
the Emperor and Camarilla had fled after a second outbreak in
Vienna—to protest against these rebellious acts. The Emperor
assured them that he had given the Ban no sanction; that he had,
indeed, dismissed him from the Imperial service. It happened that
Jellachich was at Innspruck at this very moment, carrying the
notification of dismissal in his pocket, and in his mind an unwritten
commission to serve Austria against Hungary.
In that 11th of July speech, at least, we, too, after long years, can
feel the glow. The occasion itself was dramatic. Every deputy
realized that the crisis of the revolution was at hand,—that Hungary
must either turn back, or dare to plunge into an unknown and
perilous sea. All were waiting for the decisive word.
Internal discord alone tarnished the record of the last days of the
Hungarian Republic. On July 1, Kossuth removed Görgei for
insubordination, but Görgei’s officers and men protested so loudly
that Kossuth thought it discreet to reinstate him. Three weeks later,
a fraction of the Diet, assembled at Szegedin, declared the equality
of all the races in Hungary, emancipated the Jews, and then, warned
by the rumble of hostile cannon, it dissolved forever.
For yet a few weeks we have news of Kossuth hurrying hither and
thither to proclaim hope where no hope was; conferring with
nonplussed but still resolute generals; dragging after him, like his
shadow, those printing-presses for bank-notes, now worth no more
than blank paper. Finally, at Arád, he resigned the presidency, and
appointed Görgei dictator with full powers. At Világos, on August 13,
Görgei surrendered his exhausted army of twenty-three thousand
men to Rüdiger, the Russian general. Thus was consummated what
the Magyars, frenzied by defeat, branded as Görgei’s treason, but
what, to an impartial observer, appears an inevitable act. Görgei’s
course throughout the war cannot be commended: inordinate
personal ambition, not treason, was its motive; he may have thought
to play the part of Monk, but more likely he had taken Napoleon for
his model; one thing alone is certain,—he did not intend that
Kossuth should reap the glory of victory, if victory came. In
surrendering at Világos he did what every commander is justified in
doing, when further resistance could only entail fresh losses without
any hope of altering the result.
Learning the capitulation of the main army, the other generals one
by one submitted. Klapka alone maintained an heroic defense at
Comorn until September 27, when hunger and an empty magazine
forced him to surrender. With the hauling down of the red-white-
and-green flag from the citadel of Comorn vanished the last symbol
of that revolution which, bursting forth at Palermo in January, 1848,
had spread through Europe, shaking the thrones of monarchs, and
kindling in down-trodden people the belief that a new epoch, a
Golden Age of Liberty, had come. Hopes as splendid as men ever
cherished had now been shattered, and in their stead only the
bitterest memories remained; for as each people pondered in sorrow
and oppression the events of those twenty months, it was tormented
by the reflection that its own dissensions, not less than the might of
its enemies, had wrought its ruin.