(eBook PDF) Hands-On Machine Learning with Scikit-Learn and TensorFlow instant download
(eBook PDF) Hands-On Machine Learning with Scikit-Learn and TensorFlow instant download
https://ebooksecure.com/product/ebook-pdf-hands-on-machine-
learning-with-scikit-learn-and-tensorflow/
https://ebooksecure.com/download/visualization-techniques-for-
climate-change-with-machine-learning-and-artificial-intelligence-
ebook-pdf/
http://ebooksecure.com/product/ebook-pdf-introduction-to-machine-
learning-with-python-a-guide-for-data-scientists/
http://ebooksecure.com/product/ebook-pdf-machine-learning-
refined-foundations-algorithms-and-applications/
http://ebooksecure.com/product/ebook-pdf-hands-on-virtual-
computing-2nd-edition/
(eBook PDF) Hands-On Ethical Hacking and Network
Defense 3rd Edition
http://ebooksecure.com/product/ebook-pdf-hands-on-ethical-
hacking-and-network-defense-3rd-edition/
https://ebooksecure.com/download/applied-machine-learning-ebook-
pdf/
https://ebooksecure.com/download/machine-learning-for-planetary-
science-ebook-pdf/
http://ebooksecure.com/product/ebook-pdf-hands-on-database-2nd-
edition-by-steve-conger/
http://ebooksecure.com/product/ebook-pdf-essentials-of-marketing-
research-a-hands-on-orientation/
Objective and Approach
This book assumes that you know close to nothing about Machine Learning. Its goal is to give you the
concepts, the intuitions, and the tools you need to actually implement programs capable of learning from
data.
We will cover a large number of techniques, from the simplest and most commonly used (such as linear
regression) to some of the Deep Learning techniques that regularly win competitions.
Rather than implementing our own toy versions of each algorithm, we will be using actual production-
ready Python frameworks:
Scikit-Learn is very easy to use, yet it implements many Machine Learning algorithms efficiently, so
it makes for a great entry point to learn Machine Learning.
TensorFlow is a more complex library for distributed numerical computation using data flow graphs.
It makes it possible to train and run very large neural networks efficiently by distributing the
computations across potentially thousands of multi-GPU servers. TensorFlow was created at Google
and supports many of their large-scale Machine Learning applications. It was open-sourced in
November 2015.
The book favors a hands-on approach, growing an intuitive understanding of Machine Learning through
concrete working examples and just a little bit of theory. While you can read this book without picking up
your laptop, we highly recommend you experiment with the code examples available online as Jupyter
notebooks at https://github.com/ageron/handson-ml.
Prerequisites
This book assumes that you have some Python programming experience and that you are familiar with
Python’s main scientific libraries, in particular NumPy, Pandas, and Matplotlib.
Also, if you care about what’s under the hood you should have a reasonable understanding of college-
level math as well (calculus, linear algebra, probabilities, and statistics).
If you don’t know Python yet, http://learnpython.org/ is a great place to start. The official tutorial on
python.org is also quite good.
If you have never used Jupyter, Chapter 2 will guide you through installation and the basics: it is a great
tool to have in your toolbox.
If you are not familiar with Python’s scientific libraries, the provided Jupyter notebooks include a few
tutorials. There is also a quick math tutorial for linear algebra.
Roadmap
This book is organized in two parts. Part I, The Fundamentals of Machine Learning, covers the
following topics:
What is Machine Learning? What problems does it try to solve? What are the main categories and
fundamental concepts of Machine Learning systems?
The main challenges of Machine Learning, in particular underfitting and overfitting (the
bias/variance tradeoff).
Reducing the dimensionality of the training data to fight the curse of dimensionality.
The most common learning algorithms: Linear and Polynomial Regression, Logistic Regression, k-
Nearest Neighbors, Support Vector Machines, Decision Trees, Random Forests, and Ensemble
methods.
Part II, Neural Networks and Deep Learning, covers the following topics:
What are neural nets? What are they good for?
The most important neural net architectures: feedforward neural nets, convolutional nets, recurrent
nets, long short-term memory (LSTM) nets, and autoencoders.
Reinforcement learning.
The first part is based mostly on Scikit-Learn while the second part uses TensorFlow.
CAUTION
Don’t jump into deep waters too hastily: while Deep Learning is no doubt one of the most exciting areas in Machine Learning,
you should master the fundamentals first. Moreover, most problems can be solved quite well using simpler techniques such as
Random Forests and Ensemble methods (discussed in Part I). Deep Learning is best suited for complex problems such as image
recognition, speech recognition, or natural language processing, provided you have enough data, computing power, and patience.
Other Resources
Many resources are available to learn about Machine Learning. Andrew Ng’s ML course on Coursera and
Geoffrey Hinton’s course on neural networks and Deep Learning are amazing, although they both require a
significant time investment (think months).
There are also many interesting websites about Machine Learning, including of course Scikit-Learn’s
exceptional User Guide. You may also enjoy Dataquest, which provides very nice interactive tutorials,
and ML blogs such as those listed on Quora. Finally, the Deep Learning website has a good list of
resources to learn more.
Of course there are also many other introductory books about Machine Learning, in particular:
Joel Grus, Data Science from Scratch (O’Reilly). This book presents the fundamentals of Machine
Learning, and implements some of the main algorithms in pure Python (from scratch, as the name
suggests).
Stephen Marsland, Machine Learning: An Algorithmic Perspective (Chapman and Hall). This book
is a great introduction to Machine Learning, covering a wide range of topics in depth, with code
examples in Python (also from scratch, but using NumPy).
Sebastian Raschka, Python Machine Learning (Packt Publishing). Also a great introduction to
Machine Learning, this book leverages Python open source libraries (Pylearn 2 and Theano).
Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, Learning from Data
(AMLBook). A rather theoretical approach to ML, this book provides deep insights, in particular on
the bias/variance tradeoff (see Chapter 4).
Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 3rd Edition
(Pearson). This is a great (and huge) book covering an incredible amount of topics, including
Machine Learning. It helps put ML into perspective.
Finally, a great way to learn is to join ML competition websites such as Kaggle.com this will allow you
to practice your skills on real-world problems, with help and insights from some of the best ML
professionals out there.
Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width
Used for program listings, as well as within paragraphs to refer to program elements such as
variable or function names, databases, data types, environment variables, statements and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values determined by context.
TIP
This element signifies a tip or suggestion.
NOTE
This element signifies a general note.
WARNING
This element indicates a warning or caution.
Using Code Examples
Supplemental material (code examples, exercises, etc.) is available for download at
https://github.com/ageron/handson-ml.
This book is here to help you get your job done. In general, if example code is offered with this book, you
may use it in your programs and documentation. You do not need to contact us for permission unless
you’re reproducing a significant portion of the code. For example, writing a program that uses several
chunks of code from this book does not require permission. Selling or distributing a CD-ROM of
examples from O’Reilly books does require permission. Answering a question by citing this book and
quoting example code does not require permission. Incorporating a significant amount of example code
from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher,
and ISBN. For example: “Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien
Géron (O’Reilly). Copyright 2017 Aurélien Géron, 978-1-491-96229-9.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to
contact us at permissions@oreilly.com.
O’Reilly Safari
NOTE
Safari (formerly Safari Books Online) is a membership-based training and reference platform for
enterprise, government, educators, and individuals.
Members have access to thousands of books, training videos, Learning Paths, interactive tutorials, and
curated playlists from over 250 publishers, including O’Reilly Media, Harvard Business Review,
Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press,
Adobe, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks,
Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and
Course Technology, among others.
For more information, please visit http://oreilly.com/safari.
How to Contact Us
Please address comments and questions concerning this book to the publisher:
O’Reilly Media, Inc.
Sebastopol, CA 95472
707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional information. You
can access this page at http://bit.ly/hands-on-machine-learning-with-scikit-learn-and-tensorflow.
To comment or ask technical questions about this book, send email to bookquestions@oreilly.com.
For more information about our books, courses, conferences, and news, see our website at
http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia
Acknowledgments
I would like to thank my Google colleagues, in particular the YouTube video classification team, for
teaching me so much about Machine Learning. I could never have started this project without them.
Special thanks to my personal ML gurus: Clément Courbet, Julien Dubois, Mathias Kende, Daniel
Kitachewsky, James Pack, Alexander Pak, Anosh Raj, Vitor Sessak, Wiktor Tomczak, Ingrid von Glehn,
Rich Washington, and everyone at YouTube Paris.
I am incredibly grateful to all the amazing people who took time out of their busy lives to review my book
in so much detail. Thanks to Pete Warden for answering all my TensorFlow questions, reviewing Part II,
providing many interesting insights, and of course for being part of the core TensorFlow team. You should
definitely check out his blog! Many thanks to Lukas Biewald for his very thorough review of Part II: he
left no stone unturned, tested all the code (and caught a few errors), made many great suggestions, and his
enthusiasm was contagious. You should check out his blog and his cool robots! Thanks to Justin Francis,
who also reviewed Part II very thoroughly, catching errors and providing great insights, in particular in
Chapter 16. Check out his posts on TensorFlow!
Huge thanks as well to David Andrzejewski, who reviewed Part I and provided incredibly useful
feedback, identifying unclear sections and suggesting how to improve them. Check out his website!
Thanks to Grégoire Mesnil, who reviewed Part II and contributed very interesting practical advice on
training neural networks. Thanks as well to Eddy Hung, Salim Sémaoune, Karim Matrah, Ingrid von
Glehn, Iain Smears, and Vincent Guilbeau for reviewing Part I and making many useful suggestions. And I
also wish to thank my father-in-law, Michel Tessier, former mathematics teacher and now a great
translator of Anton Chekhov, for helping me iron out some of the mathematics and notations in this book
and reviewing the linear algebra Jupyter notebook.
And of course, a gigantic “thank you” to my dear brother Sylvain, who reviewed every single chapter,
tested every line of code, provided feedback on virtually every section, and encouraged me from the first
line to the last. Love you, bro!
Many thanks as well to O’Reilly’s fantastic staff, in particular Nicole Tache, who gave me insightful
feedback, always cheerful, encouraging, and helpful. Thanks as well to Marie Beaugureau, Ben Lorica,
Mike Loukides, and Laurel Ruma for believing in this project and helping me define its scope. Thanks to
Matt Hacker and all of the Atlas team for answering all my technical questions regarding formatting,
asciidoc, and LaTeX, and thanks to Rachel Monaghan, Nick Adams, and all of the production team for
their final review and their hundreds of corrections.
Last but not least, I am infinitely grateful to my beloved wife, Emmanuelle, and to our three wonderful
kids, Alexandre, Rémi, and Gabrielle, for encouraging me to work hard on this book, asking many
questions (who said you can’t teach neural networks to a seven-year-old?), and even bringing me cookies
and coffee. What more can one dream of?
When most people hear “Machine Learning,” they picture a robot: a dependable butler or a deadly
Terminator depending on who you ask. But Machine Learning is not just a futuristic fantasy, it’s already
here. In fact, it has been around for decades in some specialized applications, such as Optical Character
Recognition (OCR). But the first ML application that really became mainstream, improving the lives of
hundreds of millions of people, took over the world back in the 1990s: it was the spam filter. Not exactly
a self-aware Skynet, but it does technically qualify as Machine Learning (it has actually learned so well
that you seldom need to flag an email as spam anymore). It was followed by hundreds of ML applications
that now quietly power hundreds of products and features that you use regularly, from better
recommendations to voice search.
Where does Machine Learning start and where does it end? What exactly does it mean for a machine to
learn something? If I download a copy of Wikipedia, has my computer really “learned” something? Is it
suddenly smarter? In this chapter we will start by clarifying what Machine Learning is and why you may
want to use it.
Then, before we set out to explore the Machine Learning continent, we will take a look at the map and
learn about the main regions and the most notable landmarks: supervised versus unsupervised learning,
online versus batch learning, instance-based versus model-based learning. Then we will look at the
workflow of a typical ML project, discuss the main challenges you may face, and cover how to evaluate
and fine-tune a Machine Learning system.
This chapter introduces a lot of fundamental concepts (and jargon) that every data scientist should know
by heart. It will be a high-level overview (the only chapter without much code), all rather simple, but you
should make sure everything is crystal-clear to you before continuing to the rest of the book. So grab a
coffee and let’s get started!
TIP
If you already know all the Machine Learning basics, you may want to skip directly to Chapter 2. If you are not sure, try to
answer all the questions listed at the end of the chapter before moving on.
What Is Machine Learning?
Machine Learning is the science (and art) of programming computers so they can learn from data.
Here is a slightly more general definition:
[Machine Learning is the] field of study that gives computers the ability to learn without being
explicitly programmed.
Arthur Samuel, 1959
And a more engineering-oriented one:
A computer program is said to learn from experience E with respect to some task T and some
performance measure P, if its performance on T, as measured by P, improves with experience E.
Tom Mitchell, 1997
For example, your spam filter is a Machine Learning program that can learn to flag spam given examples
of spam emails (e.g., flagged by users) and examples of regular (nonspam, also called “ham”) emails. The
examples that the system uses to learn are called the training set. Each training example is called a
training instance (or sample). In this case, the task T is to flag spam for new emails, the experience E is
the training data, and the performance measure P needs to be defined; for example, you can use the ratio
of correctly classified emails. This particular performance measure is called accuracy and it is often
used in classification tasks.
If you just download a copy of Wikipedia, your computer has a lot more data, but it is not suddenly better
at any task. Thus, it is not Machine Learning.
Why Use Machine Learning?
Consider how you would write a spam filter using traditional programming techniques (Figure 1-1):
1. First you would look at what spam typically looks like. You might notice that some words or
phrases (such as “4U,” “credit card,” “free,” and “amazing”) tend to come up a lot in the subject.
Perhaps you would also notice a few other patterns in the sender’s name, the email’s body, and
so on.
2. You would write a detection algorithm for each of the patterns that you noticed, and your
program would flag emails as spam if a number of these patterns are detected.
3. You would test your program, and repeat steps 1 and 2 until it is good enough.
Since the problem is not trivial, your program will likely become a long list of complex rules — pretty
hard to maintain.
In contrast, a spam filter based on Machine Learning techniques automatically learns which words and
phrases are good predictors of spam by detecting unusually frequent patterns of words in the spam
examples compared to the ham examples (Figure 1-2). The program is much shorter, easier to maintain,
and most likely more accurate.
Figure 1-2. Machine Learning approach
Moreover, if spammers notice that all their emails containing “4U” are blocked, they might start writing
“For U” instead. A spam filter using traditional programming techniques would need to be updated to flag
“For U” emails. If spammers keep working around your spam filter, you will need to keep writing new
rules forever.
In contrast, a spam filter based on Machine Learning techniques automatically notices that “For U” has
become unusually frequent in spam flagged by users, and it starts flagging them without your intervention
(Figure 1-3).
Figure 1-3. Automatically adapting to change
Another area where Machine Learning shines is for problems that either are too complex for traditional
approaches or have no known algorithm. For example, consider speech recognition: say you want to start
simple and write a program capable of distinguishing the words “one” and “two.” You might notice that
the word “two” starts with a high-pitch sound (“T”), so you could hardcode an algorithm that measures
high-pitch sound intensity and use that to distinguish ones and twos. Obviously this technique will not
scale to thousands of words spoken by millions of very different people in noisy environments and in
dozens of languages. The best solution (at least today) is to write an algorithm that learns by itself, given
many example recordings for each word.
Finally, Machine Learning can help humans learn (Figure 1-4): ML algorithms can be inspected to see
what they have learned (although for some algorithms this can be tricky). For instance, once the spam
filter has been trained on enough spam, it can easily be inspected to reveal the list of words and
combinations of words that it believes are the best predictors of spam. Sometimes this will reveal
unsuspected correlations or new trends, and thereby lead to a better understanding of the problem.
Applying ML techniques to dig into large amounts of data can help discover patterns that were not
immediately apparent. This is called data mining.
Figure 1-4. Machine Learning can help humans learn
Complex problems for which there is no good solution at all using a traditional approach: the best
Machine Learning techniques can find a solution.
Whether or not they can learn incrementally on the fly (online versus batch learning)
Whether they work by simply comparing new data points to known data points, or instead detect
patterns in the training data and build a predictive model, much like scientists do (instance-based
versus model-based learning)
These criteria are not exclusive; you can combine them in any way you like. For example, a state-of-the-
art spam filter may learn on the fly using a deep neural network model trained using examples of spam and
ham; this makes it an online, model-based, supervised learning system.
Let’s look at each of these criteria a bit more closely.
Another Random Document on
Scribd Without Any Related Topics
damaged disk or other medium, a computer virus, or computer
codes that damage or cannot be read by your equipment.
1.F.4. Except for the limited right of replacement or refund set forth
in paragraph 1.F.3, this work is provided to you ‘AS-IS’, WITH NO
OTHER WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.
Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.
Most people start at our website which has the main PG search
facility: www.gutenberg.org.
ebooksecure.com