Explore the full ebook collection and download it now at textbookfull.

com

Learning Probabilistic Graphical Models in R 1st

Edition David Bellot

https://textbookfull.com/product/learning-probabilistic-
graphical-models-in-r-1st-edition-david-bellot/

OR CLICK HERE

DOWLOAD EBOOK

Browse and Get More Ebook Downloads Instantly at https://textbookfull.com

Click here to visit textbookfull.com and download textbook now
 Your digital treasures (PDF, ePub, MOBI) await
Download instantly and pick your perfect format...

Read anywhere, anytime, on any device!

Reasoning with Probabilistic and Deterministic Graphical

Models Exact Algorithms Second Edition Rina Dechter

https://textbookfull.com/product/reasoning-with-probabilistic-and-
deterministic-graphical-models-exact-algorithms-second-edition-rina-
dechter/
textbookfull.com

Graphical Simulation of Deformable Models 1st Edition

Jianping Cai

https://textbookfull.com/product/graphical-simulation-of-deformable-
models-1st-edition-jianping-cai/

textbookfull.com

Graphical Data Analysis with R Antony Unwin

https://textbookfull.com/product/graphical-data-analysis-with-r-
antony-unwin/

textbookfull.com

Probabilistic Deep Learning with Python 1st Edition Oliver

Duerr

https://textbookfull.com/product/probabilistic-deep-learning-with-
python-1st-edition-oliver-duerr/

textbookfull.com
Reliability Engineering Probabilistic Models and
Maintenance Methods Second Edition Nachlas

https://textbookfull.com/product/reliability-engineering-
probabilistic-models-and-maintenance-methods-second-edition-nachlas/

textbookfull.com

Reliability engineering. Probabilistic models and

maintenance methods Second Edition Nachlas

https://textbookfull.com/product/reliability-engineering-
probabilistic-models-and-maintenance-methods-second-edition-nachlas-2/

textbookfull.com

Introduction to Deep Learning Using R: A Step-by-Step

Guide to Learning and Implementing Deep Learning Models
Using R Taweh Beysolow Ii
https://textbookfull.com/product/introduction-to-deep-learning-using-
r-a-step-by-step-guide-to-learning-and-implementing-deep-learning-
models-using-r-taweh-beysolow-ii/
textbookfull.com

Advanced R Statistical Programming and Data Models:

Analysis, Machine Learning, and Visualization 1st Edition
Matt Wiley
https://textbookfull.com/product/advanced-r-statistical-programming-
and-data-models-analysis-machine-learning-and-visualization-1st-
edition-matt-wiley/
textbookfull.com

Machine Learning with R Cookbook Second Edition Analyze

data and build predictive models Bhatia

https://textbookfull.com/product/machine-learning-with-r-cookbook-
second-edition-analyze-data-and-build-predictive-models-bhatia/

textbookfull.com
Learning Probabilistic Graphical
Models in R

Familiarize yourself with probabilistic graphical models

through real-world problems and illustrative code
examples in R

David Bellot

BIRMINGHAM - MUMBAI
Learning Probabilistic Graphical Models in R

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book is
sold without warranty, either express or implied. Neither the author, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages
caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

First published: April 2016

Production reference: 1270416

Published by Packt Publishing Ltd.

Livery Place
35 Livery Street
Birmingham B3 2PB, UK.

ISBN 978-1-78439-205-5

www.packtpub.com
 Credits

Author Project Coordinator

David Bellot Kinjal Bari

Reviewers Proofreader
Mzabalazo Z. Ngwenya Safis Editing
Prabhanjan Tattar
Indexer
Acquisition Editor Mariammal Chettiyar
Divya Poojari
Graphics
Content Development Editor Abhinash Sahu
Trusha Shriyan
Production Coordinator
Technical Editor Nilesh Mohite
Vivek Arora
Cover Work
Copy Editor Nilesh Mohite
Stephen Copestake
 About the Author

David Bellot is a PhD graduate in computer science from INRIA, France, with a
focus on Bayesian machine learning. He was a postdoctoral fellow at the University
of California, Berkeley, and worked for companies such as Intel, Orange, and
Barclays Bank. He currently works in the financial industry, where he develops
financial market prediction algorithms using machine learning. He is also a
contributor to open source projects such as the Boost C++ library.
 About the Reviewers

Mzabalazo Z. Ngwenya holds a postgraduate degree in mathematical statistics

from the University of Cape Town. He has worked extensively in the field of
statistical consulting and has considerable experience working with R. Areas of
interest to him are primarily centered around statistical computing. Previously,
he has been involved in reviewing the following Packt Publishing titles: Learning
RStudio for R Statistical Computing, Mark P.J. van der Loo and Edwin de Jonge;
R Statistical Application Development by Example Beginner's Guide, Prabhanjan
Narayanachar Tattar; Machine Learning with R, Brett Lantz; R Graph Essentials, David
Alexandra Lillis; R Object-oriented Programming, Kelly Black; Mastering Scientific
Computing with R, Paul Gerrard and Radia Johnson; and Mastering Data Analysis with
R, Gergely Darócz.

Prabhanjan Tattar is currently working as a senior data scientist at Fractal Analytics,

Inc. He has 8 years of experience as a statistical analyst. Survival analysis and statistical
inference are his main areas of research/interest. He has published several research
papers in peer-reviewed journals and authored two books on R: R Statistical Application
Development by Example, Packt Publishing; and A Course in Statistics with R, Wiley. The
R packages gpk, RSADBE, and ACSWR are also maintained by him.
 www.PacktPub.com

eBooks, discount offers, and more

Did you know that Packt offers eBook versions of every book published, with PDF
and ePub files available? You can upgrade to the eBook version at www.PacktPub.
com and as a print book customer, you are entitled to a discount on the eBook copy.
Get in touch with us at customercare@packtpub.com for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign
up for a range of free newsletters and receive exclusive discounts and offers on Packt
books and eBooks.
TM

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital
book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?
• Fully searchable across every book published by Packt
• Copy and paste, print, and bookmark content
• On demand and accessible via a web browser
 Table of Contents
Preface v
Chapter 1: Probabilistic Reasoning 1
Machine learning 4
Representing uncertainty with probabilities 5
Beliefs and uncertainty as probabilities 6
Conditional probability 7
Probability calculus and random variables 7
Sample space, events, and probability 7
Random variables and probability calculus 8
Joint probability distributions 10
Bayes' rule 11
Interpreting the Bayes' formula 13
A first example of Bayes' rule 13
A first example of Bayes' rule in R 16
Probabilistic graphical models 20
Probabilistic models 20
Graphs and conditional independence 21
Factorizing a distribution 23
Directed models 24
Undirected models 25
Examples and applications 26
Summary 31
Chapter 2: Exact Inference 33
Building graphical models 35
Types of random variable 36
Building graphs 37
Probabilistic expert system 37
Basic structures in probabilistic graphical models 40
Variable elimination 44

[i]
Table of Contents

Sum-product and belief updates 47

The junction tree algorithm 51
Examples of probabilistic graphical models 62
The sprinkler example 62
The medical expert system 63
Models with more than two layers 64
Tree structure 66
Summary 68
Chapter 3: Learning Parameters 69
Introduction 71
Learning by inference 75
Maximum likelihood 79
How are empirical and model distribution related? 79
The ML algorithm and its implementation in R 82
Application 86
Learning with hidden variables – the
EM algorithm 88
Latent variables 89
Principles of the EM algorithm 90
Derivation of the EM algorithm 91
Applying EM to graphical models 93
Summary 94
Chapter 4: Bayesian Modeling – Basic Models 97
The Naive Bayes model 98
Representation 100
Learning the Naive Bayes model 101
Bayesian Naive Bayes 104
Beta-Binomial 106
The prior distribution 111
The posterior distribution with the conjugacy property 112
Which values should we choose for the Beta parameters? 113
The Gaussian mixture model 115
Definition 116
Summary 122
Chapter 5: Approximate Inference 125
Sampling from a distribution 126
Basic sampling algorithms 129
Standard distributions 129
Rejection sampling 133
An implementation in R 135

[ ii ]
 Table of Contents

Importance sampling 142

An implementation in R 144
Markov Chain Monte-Carlo 152
General idea of the method 153
The Metropolis-Hastings algorithm 154
MCMC for probabilistic graphical models in R 162
Installing Stan and RStan 163
A simple example in RStan 164
Summary 165
Chapter 6: Bayesian Modeling – Linear Models 167
Linear regression 169
Estimating the parameters 170
Bayesian linear models 176
Over-fitting a model 176
Graphical model of a linear model 179
Posterior distribution 181
Implementation in R 184
A stable implementation 188
More packages in R 194
Summary 195
Chapter 7: Probabilistic Mixture Models 197
Mixture models 198
EM for mixture models 200
Mixture of Bernoulli 207
Mixture of experts 210
Latent Dirichlet Allocation 215
The LDA model 216
Variational inference 220
Examples 221
Summary 224
Appendix 227
References 227
Books on the Bayesian theory 227
Books on machine learning 228
Papers 228
Index 229

[ iii ]
 Preface
Probabilistic graphical models is one of the most advanced techniques in machine
learning to represent data and models in the real world with probabilities. In many
instances, it uses the Bayesian paradigm to describe algorithms that can draw
conclusions from noisy and uncertain real-world data.

The book covers topics such as inference (automated reasoning and learning), which
is automatically building models from raw data. It explains how all the algorithms
work step by step and presents readily usable solutions in R with many examples.
After covering the basic principles of probabilities and the Bayes formula, it presents
Probabilistic Graphical Models(PGMs) and several types of inference and learning
algorithms. The reader will go from the design to the automatic fitting of the model.

Then, the books focuses on useful models that have proven track records in solving
many data science problems, such as Bayesian classifiers, Mixtures models, Bayesian
Linear Regression, and also simpler models that are used as basic components to
build more complex models.

What this book covers

Chapter 1, Probabilistic Reasoning, covers topics from the basic concepts of probabilities
to PGMs as a generic framework to do tractable, efficient, and easy modeling with
probabilistic models, through the presentation of the Bayes formula.

Chapter 2, Exact Inference, shows you how to build PGMs by combining simple
graphs and perform queries on the model using an exact inference algorithm called
the junction tree algorithm.

Chapter 3, Learning Parameters, includes fitting and learning the PGM models from
data sets with the Maximum Likelihood approach.

[v]
Preface

Chapter 4, Bayesian Modeling – Basic Models, covers simple and powerful Bayesian
models that can be used as building blocks for more advanced models and shows
you how to fit and query them with adapted algorithms.

Chapter 5, Approximate Inference, covers the second way to perform an inference in

PGM using sampling algorithms and a presentation of the main sampling algorithms
such as MCMC.

Chapter 6, Bayesian Modeling – Linear Models, shows you a more Bayesian view of the
standard linear regression algorithm and a solution to the problem of over-fitting.

Chapter 7, Probabilistic Mixture Models, goes over more advanced probabilistic models
in which the data comes from a mixture of several simple models.

Appendix, References, includes all the books and articles which have been used to
write this book.

What you need for this book

All the examples in this book can be used with R version 3 or above on any platform
and operating system supporting R.

Who this book is for

This book is for anyone who has to deal with lots of data and draw conclusions from
it, especially when the data is noisy or uncertain. Data scientists, machine learning
enthusiasts, engineers, and those who are curious about the latest advances in
machine learning will find PGM interesting.

Conventions
In this book, you will find a number of text styles that distinguish between different
kinds of information. Here are some examples of these styles and an explanation of
their meaning.

Code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles are shown as follows:
"We can also mention the arm package, which provides Bayesian versions of glm()
and polr() and implements hierarchical models."

[ vi ]
 Preface

Any command-line input or output is written as follows:

pred_sigma <- sqrt(sigma^2 + apply((T%*%posterior_sigma)*T, MARGIN=1,
FUN=sum))
upper_bound <- T%*%posterior_beta + qnorm(0.95)*pred_sigma
lower_bound <- T%*%posterior_beta - qnorm(0.95)*pred_sigma

Warnings or important notes appear in a box like this.

Tips and tricks appear like this.

Reader feedback
Feedback from our readers is always welcome. Let us know what you think about
this book—what you liked or disliked. Reader feedback is important for us as it helps
us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail feedback@packtpub.com, and mention

the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing
or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support
Now that you are the proud owner of a Packt book, we have a number of things to
help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at
http://www.packtpub.com. If you purchased this book elsewhere, you can visit
http://www.packtpub.com/support and register to have the files e-mailed directly
to you.

[ vii ]
Preface

You can download the code files by following these steps:

1. Log in or register to our website using your e-mail address and password.
2. Hover the mouse pointer on the SUPPORT tab at the top.
3. Click on Code Downloads & Errata.
4. Enter the name of the book in the Search box.
5. Select the book for which you're looking to download the code files.
6. Choose from the drop-down menu where you purchased this book from.
7. Click on Code Download.

You can also download the code files by clicking on the Code Files button on the
book's webpage at the Packt Publishing website. This page can be accessed by
entering the book's name in the Search box. Please note that you need to be logged in
to your Packt account.

Once the file is downloaded, please make sure that you unzip or extract the folder
using the latest version of:

• WinRAR / 7-Zip for Windows

• Zipeg / iZip / UnRarX for Mac
• 7-Zip / PeaZip for Linux

Errata
Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you find a mistake in one of our books—maybe a mistake in the text or
the code—we would be grateful if you could report this to us. By doing so, you can
save other readers from frustration and help us improve subsequent versions of this
book. If you find any errata, please report them by visiting http://www.packtpub.
com/submit-errata, selecting your book, clicking on the Errata Submission Form
link, and entering the details of your errata. Once your errata are verified, your
submission will be accepted and the errata will be uploaded to our website or added
to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/

content/support and enter the name of the book in the search field. The required
information will appear under the Errata section.

[ viii ]
 Preface

Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all
media. At Packt, we take the protection of our copyright and licenses very seriously.
If you come across any illegal copies of our works in any form on the Internet, please
provide us with the location address or website name immediately so that we can
pursue a remedy.

Please contact us at copyright@packtpub.com with a link to the suspected pirated

material.

We appreciate your help in protecting our authors and our ability to bring you
valuable content.

Questions
If you have a problem with any aspect of this book, you can contact us at
questions@packtpub.com, and we will do our best to address the problem.

[ ix ]
 Probabilistic Reasoning
Among all the predictions that were made about the 21st century, maybe the most
unexpected one was that we would collect such a formidable amount of data about
everything, everyday, and everywhere in the world. Recent years have seen an
incredible explosion of data collection about our world, our lives, and technology;
this is the main driver of what we can certainly call a revolution. We live in the Age
of Information. But collecting data is nothing if we don't exploit it and try to extract
knowledge out of it.

At the beginning of the 20th century, with the birth of statistics, the world was all about
collecting data and making statistics. In that time, the only reliable tools were pencils
and paper and of course, the eyes and ears of the observers. Scientific observation was
still in its infancy, despite the prodigious development of the 19th century.

More than a hundred years later, we have computers, we have electronic sensors,
we have massive data storage and we are able to store huge amounts of data
continuously about, not only our physical world, but also our lives, mainly through
the use of social networks, the Internet, and mobile phones. Moreover, the density of
our storage technology has increased so much that we can, nowadays, store months
if not years of data into a very small volume that can fit in the palm of our hand.

But storing data is not acquiring knowledge. Storing data is just keeping it
somewhere for future use. At the same time as our storage capacity dramatically
evolved, the capacity of modern computers increased too, at a pace that is sometimes
hard to believe. When I was a doctoral student, I remember how proud I was when
in the laboratory I received that brand-new, shiny, all-powerful PC for carrying my
research work. Today, my old smart phone, which fits in my pocket, is more than 20
times faster.

[1]
Probabilistic Reasoning

Therefore in this book, you will learn one of the most advanced techniques to
transform data into knowledge: machine learning. This technology is used in every
aspect of modern life now, from search engines, to stock market predictions, from
speech recognition to autonomous vehicles. Moreover it is used in many fields
where one would not suspect it at all, from quality assurance in product chains to
optimizing the placement of antennas for mobile phone networks.

Machine learning is the marriage between computer science and probabilities and
statistics. A central theme in machine learning is the problem of inference or how to
produce knowledge or predictions using an algorithm fed with data and examples.
And this brings us to the two fundamental aspects of machine learning: the design of
algorithms that can extract patterns and high-level knowledge from vast amounts of
data and also the design of algorithms that can use this knowledge—or, in scientific
terms: learning and inference.

Pierre-Simon Laplace (1749-1827) a French mathematician and one of the greatest

scientists of all time, was presumably among the first to understand an important
aspect of data collection: data is unreliable, uncertain and, as we say today, noisy.
He was also the first to develop the use of probabilities to deal with such aspects of
uncertainty and to represent one's degree of belief about an event or information.

In his Essai philosophique sur les probabilités (1814), Laplace formulated an original
mathematical system for reasoning about new and old data, in which one's belief
about something could be updated and improved as soon as new data where
available. Today we call that Bayesian reasoning. Indeed Thomas Bayes was
the first, toward the end of the 18th century, to discover this principle. Without
any knowledge about Bayes' work, Pierre-Simon Laplace rediscovered the same
principle and formulated the modern form of the Bayes theorem. It is interesting
to note that Laplace eventually learned about Bayes' posthumous publications
and acknowledged Bayes to be the first to describe the principle of this inductive
reasoning system. Today, we speak about Laplacian reasoning instead of Bayesian
reasoning and we call it the Bayes-Price-Laplace theorem.

More than a century later, this mathematical technique was reborn thanks to new
discoveries in computing probabilities and gave birth to one of the most important
and used techniques in machine learning: the probabilistic graphical model.

From now on, it is important to note that the term graphical refers to the theory of
graphs—that is, a mathematical object with nodes and edges (and not graphics or
drawings). You know that, when you want to explain to someone the relationships
between different objects or entities, you take a sheet of paper and draw boxes that
you connect with lines or arrows. It is an easy and neat way to show relationships,
whatever they are, between different elements.

[2]
 Chapter 1

Probabilistic Graphical Models (PGM for short) are exactly that: you want to
describe relationships between variables. However, you don't have any certainty
about your variables, but rather beliefs or uncertain knowledge. And we know now
that probabilities are the way to represent and deal with such uncertainties, in a
mathematical and rigorous way.

A probabilistic graphical model is a tool to represent beliefs and uncertain knowledge

about facts and events using probabilities. It is also one of the most advanced machine
learning techniques nowadays and has many industrial success stories.

Probabilistic graphical models can deal with our imperfect knowledge about the
world because our knowledge is always limited. We can't observe everything, we
can't represent all the universe in a computer. We are intrinsically limited as human
beings, as are our computers. With probabilistic graphical models, we can build
simple learning algorithms or complex expert systems. With new data, we can
improve those models and refine them as much as we can and also we can infer new
information or make predictions about unseen situations and events.

In this first chapter you will learn about the fundamentals needed to understand
probabilistic graphical models; that is, probabilities and the simple rules of calculus on
which they are based. We will have an overview of what we can do with probabilistic
graphical models and the related R packages. These techniques are so successful that
we will have to restrict ourselves to just the most important R packages.

We will see how to develop simple models, piece by piece, like a brick game and
how to connect models together to develop even more advanced expert systems.
We will cover the following concepts and applications and each section will contain
numerical examples that you can directly use with R:

• Machine learning
• Representing uncertainty with probabilities
• Notions of probabilistic expert systems
• Representing knowledge with graphs
• Probabilistic graphical models
• Examples and applications

[3]
Probabilistic Reasoning

Machine learning
This book is about a field of science called machine learning, or more generally
artificial intelligence. To perform a task, to reach conclusions from data, a computer
as well as any living being needs to observe and process information of a diverse
nature. For a long time now, we have been designing and inventing algorithms and
systems that can solve a problem, very accurately and at incredible speed, but all
algorithms are limited to the very specific task they were designed for. On the other
hand, living beings in general and human beings (as well as many other animals)
exhibit this incredible capacity to adapt and improve using their experience, their
errors, and what they observe in the world.

Trying to understand how it is possible to learn from experience and adapt to

changing conditions has always been a great topic of science. Since the invention of
computers, one of the main goals has been to reproduce this type of skill in a machine.

Machine learning is the study of algorithms that can learn and adapt from data
and observation, reason, and perform tasks using learned models and algorithms.
As the world we live in is inherently uncertain, in the sense that even the simplest
observation such as the color of the sky is impossible to determine absolutely, we
needed a theory that can encompass this uncertainty. The most natural one is the
theory of probability, which will serve as the mathematical foundation of the
present book.

But when the amount of data grows to very large datasets, even the simplest
probabilistic tasks can become cumbersome and we need a framework that will
allow the easy development of models and algorithms that have the necessary
complexity to deal with real-world problems.

By real-world problems, we really think of tasks that a human being is able to do

such as understanding people's speech, driving a car, trading the stock exchange,
recognizing people's faces on a picture, or making a medical diagnosis.

At the beginning of artificial intelligence, building such models and algorithms was a
very complex task and, every time a new algorithm was invented, implemented, and
programmed with inherent sources of errors and bias. The framework we present
in this book, called probabilistic graphical models, aims at separating the tasks of
designing a model and implementing algorithm. Because it is based on probability
theory and graph theory, it has very strong mathematical foundations. But also, it is a
framework where the practitioner doesn't need to write and rewrite algorithms all the
time, for algorithms were designed to solve very generic problems and already exist.

Moreover, probabilistic graphical models are based on machine learning techniques

which will help the practitioner to create new models from data in the easiest way.

[4]
 Chapter 1

Algorithms in probabilistic graphical models can learn new models from data and
answer all sorts of questions using those data and the models, and of course adapt
and improve the models when new data is available.

In this book, we will also see that probabilistic graphical models are a mathematical
generalization of many standard and classical models that we all know and that we
can reuse, mix, and modify within this framework.

The rest of this chapter will introduce required notions in probabilities and graph
theory to help you understand and use probabilistic graphical models in R.

One last note about the title of the book: Learning Probabilistic Graphical Models in R.
In fact this title has two meanings: you will learn how to make probabilistic graphical
models, and you will learn how the computer can learn probabilistic graphical
models. This is machine learning!

Representing uncertainty with

probabilities
Probabilistic graphical models, seen from the point of view of mathematics, are a
way to represent a probability distribution over several variables, which is called
a joint probability distribution. In other words, it is a tool to represent numerical
beliefs in the joint occurrence of several variables. Seen like this, it looks simple, but
what PGM addresses is the representation of these kinds of probability distribution
over many variables. In some cases, many could be really a lot, such as thousands
to millions. In this section, we will review the basic notions that are fundamental to
PGMs and see their basic implementation in R. If you're already familiar with these,
you can easily skip this section. We start by asking why probabilities are a good tool
to represent one's belief about facts and events, then we will explore the basics of
probability calculus. Next we will introduce the fundamental building blocks of any
Bayesian model and do a few simple yet interesting computations. Finally, we will
speak about the main topic of this book: Bayesian inference.

Did I say Bayesian inference was the main topic before? Indeed, probabilistic
graphical models are also a state-of-the-art approach to performing Bayesian
inference or in other words to computing new facts and conclusions from your
previous beliefs and supplying new data.

This principle of updating a probabilistic model was first discovered by Thomas

Bayes and publish by his friend Richard Price in 1763 in the now famous An Essay
toward solving a Problem in the Doctrine of Chances.

[5]
Probabilistic Reasoning

Beliefs and uncertainty as probabilities

Probability theory is nothing but common sense reduced to calculation

Théorie analytique des probabilités,

1821. Pierre-Simon, marquis de Laplace

As Pierre-Simon Laplace was saying, probabilities are a tool to quantify our

common-sense reasoning and our degree of belief. It is interesting to note that, in the
context of machine learning, this concept of belief has been somehow extended to the
machine, that is, to the computer. Through the use of algorithms, the computer will
represent its belief about certain facts and events with probabilities.

So let's take a simple example that everyone knows: the game of flipping a coin.
What's the probability or the chance that coin will land on a head, or on a tail?
Everyone should and will answer, with reason, a 50% chance or a probability of 0.5
(remember, probabilities are numbered between 0 and 1).

This simple notion has two interpretations. One we will call a frequentist
interpretation and the other one a Bayesian interpretation. The first one, the
frequentist, means that, if we flip the coin many times, in the long term it will land
heads-up half of the time and tails-up the other half of the time. Using numbers,
it will have a 50% chance of landing on one side, or a probability of 0.5. However,
this frequentist concept, as the name suggests, is valid only if one can repeat the
experiment a very large number of times. Indeed, it would not make any sense
to talk about frequency if you observe a fact only once or twice. The Bayesian
interpretation, on the other hand, quantifies our uncertainty about a fact or an event
by assigning a number (between 0 and 1, or 0% and 100%) to it. If you flip a coin,
even before playing, I'm sure you will assign a 50% chance to each face. If you watch
a horse race with 10 horses and you know nothing about the horses and their rides,
you will certainly assign a probability of 0.1 (or 10%) to each horse.

Flipping a coin is an experiment you can do many times, thousands of times or more
if you want. However, a horse race is not an experiment you can repeat numerous
times. And what is the probability your favorite team will win the next football
game? It is certainly not an experiment you can do many times: in fact you will do it
once, because there is only one match. But because you strongly believe your team
is the best this year, you will assign a probability of, say, 0.9 that your team will win
the next game.

The main advantage of the Bayesian interpretation is that it does not use the notion
of long-term frequency or repetition of the same experiment.

[6]
 Chapter 1

In machine learning, probabilities are the basic components of most of the systems
and algorithms. You might want to know the probability that an e-mail you received
is a spam (junk) e-mail. You want to know the probability that the next customer on
your online site will buy the same item as the previous customer (and whether your
website should advertise it right away). You want to know the probability that, next
month, your shop will have as many customers as this month.

As you can see with these examples, the line between purely frequentist and purely
Bayesian is far from being clear. And the good news is that the rules of probability
calculus are rigorously the same, whatever interpretation you choose (or not).

Conditional probability
A central theme in machine learning and especially in probabilistic graphical
models is the notion of a conditional probability. In fact, let's be clear, probabilistic
graphical models are all about conditional probability. Let's get back to our horse
race example. We say that, if you know nothing about the riders and their horses,
you would assign, say, a probability of 0.1 to each (assuming there are 10 horses).
Now, you just learned that the best rider in the country is participating in this race.
Would you give him the same chance as the others? Certainly not! Therefore the
probability for this rider to win is, say, 19% and therefore, we will say that all other
riders have a probability to win of only 9%. This is a conditional probability: that is, a
probability of an event based on knowing the outcome of another event. This notion
of probability matches perfectly changing our minds intuitively or updating our beliefs
(in more technical terms) given a new piece of information. At the same time we also
saw a simple example of Bayesian update where we reconsidered and updated our
beliefs given a new fact. Probabilistic graphical models are all about that but just
with more complex situations.

Probability calculus and random variables

In the previous section we saw why probabilities are a good tool to represent
uncertainty or the beliefs and frequency of an event or a fact. We also mentioned the
fact that the same rules of calculus apply for both the Bayesian and the frequentist
interpretation. In this section, we will have a first look at the basic probability rules
of calculus, and introduce the notion of a random variable which is central to Bayesian
reasoning and the probabilistic graphical models.

Sample space, events, and probability

In this section we introduce the basic concepts and the language used in probability
theory that we will use throughout the book. If you already know those concepts,
you can skip this section.
[7]
Probabilistic Reasoning

A sample space Ω is the set of all possible outcomes of an experiment. In this set, we
call ω a point of Ω, a realization. And finally we call a subset of Ω an event.

For example, if we toss a coin once, we can have heads (H) or tails (T). We say that the
sample space is Ω = {H , T } . An event could be I get a head (H). If we toss the coin twice,
the sample space is bigger and we can have all those possibilities Ω = {HH , HT , TH , TT } .
An event could be I get a head first. Therefore my event is E = {HH , HT } .

A more advanced example could be the measurement of someone's height in

centimeters. The sample space is all the positive numbers from 0.0 to 10.9. Chances
are that none of your friends will be 10.9 meters tall, but it does no harm to the
theory. An event could be all the basketball players, that is, measurements that are
2 meters or more. In mathematical notation we write in terms of intervals Ω = [ 0,10.9]
and E = [ 2,10.9] .

A probability is a real number Pr(E) that we assign to every event E. A probability

must satisfy the three following axioms. Before writing them, it is time to recall why
we're using these axioms. If you remember, we said that, whatever the interpretation
of the probabilities that we make (frequentist or Bayesian), the rules governing the
calculus of probability are the same:

• For every event E, P ( E ) ≥ 0 : we just say that probability is always positive

P ( Ω ) = 1 , which means that the probability of having any of all the possible
events is always 1. Therefore, from axiom 1 and 2, any probability is always
between 0 and 1.
• If you have independent events E1, E2, … then P (U i =1Ei ) = ∑i =1 P ( Ei ) .
∞ ∞

Random variables and probability calculus

In a computer program, a variable is a name or a label associated to a storage space
somewhere in the computer's memory. A program's variable is therefore defined by
its location (and in many languages its type) and holds one and only one value. The
value can be complex like an array or a data structure. The most important thing is
that, at any time, the value is well known and not subject to change unless someone
decides to change it. In other words, the value can only change if the algorithm using
it changes it.

A random variable is something different: it is a function from a sample space into real
numbers. For example, in some experiments, random variables are implicitly used:

• When throwing two dices, X is the sum of the numbers is a random variable
• When tossing a coin N times, X is the number of heads in N tosses is a
random variable

[8]
 Chapter 1

For each possible event, we can associate a probability pi and the set of all those
probabilities is the probability distribution of the random variable.

Let's see an example: we consider an experiment in which we toss a coin three times.
A sample point (from the sample space) is the result of the three tosses. For example,
HHT, two heads and one tail, is a sample point.

Therefore, it is easy to enumerate all the possible outcomes and find that the
sample space is:

S = { HHH , HHT , HTH , THH , TTH , THT , HTT , TTT }

Let's Hi be the event that the ith toss is a head. So for example:

H1 = { HHH , HHT , HTH , HTT }

If we assign a probability of to each sample point, then using enumeration

we see that P ( H1 ) = P ( H 2 ) = P ( H 3 ) = 1 .
2
Under this probability model, the events H1, H2, H3 are mutually independent.
To verify, we first write that:

1 1 1 1
P ( H1 ∩ H 2 ∩ H 3 ) = P ({ HHH } ) = = ⋅ ⋅ = P ( H1 ) P ( H 2 ) P ( H 3 )
8 2 2 2

We must also check each pair. For example:

2 1 1
P ( H1 ∩ H 2 ) = P ({ HHH , HHT } ) = = ⋅ = P ( H1 ) P ( H 2 )
8 2 2

The same applies to the two other pairs. Therefore H1, H2, H3 are mutually
independent. In general, we write that the probability of two independent events is the
product of their probability: P ( A ∩ B ) = P ( A ) .P ( B ) . And we write that the probability of
two disjoint independent events is the sum of their probability: P ( A ∨ B ) = P ( A ) + P ( B ) .

If we consider a different outcome, we can define another probability distribution.

For example, let's consider again the experiment in which a coin is tossed three
times. This time we consider the random variable X as the number of heads obtained
after three tosses.

[9]
Probabilistic Reasoning

A complete enumeration gives the same sample space as before:

S = { HHH , HHT , HTH , THH , TTH , THT , HTT , TTT }

But as we consider the number of heads, the random variable X will map the sample
space to the following numbers this time:

s HHH HHT HTH THH TTH THT HTT TTT

X(s) 3 2 2 2 1 1 1 0

So the range for the random variable X is now {0,1,2,3}. If we assume the same
probability for all points as before, that is , then we can deduce the probability
function on the range of X:

x 0 1 2 3
P(X=x) 3 3
8 8

Joint probability distributions

Let's come back to the first game: 2 heads and a 6 simultaneously, the hard game
with a low probability of winning. We can associate to the coin toss experiment a
random variable N, which is the number of heads after two tosses. This random
variable summarizes our experiment very well and N can take the values 0, 1, or
2. So instead of saying we're interested in the event of having two heads, we can
say equivalently that we are interested in the event N=2. This approach allows us
to look at other events, such as having only one head (HT or TH) and even having
zero heads (TT). We say that the function assigning a probability to each value of N
is called a probability distribution. Another random variable is D, the number we
obtain when we throw a dice.

When we consider the two experiments together (tossing a coin twice and throwing
a dice), we are interested in the probability of obtaining either 0, 1, or 2 heads and
at the same time obtaining either 1, 2, 3, 4, 5, or 6 with the dice. The probability
distribution of these two random variables considered at the same time is written
P(N, D) and it is called a joint probability distribution.

[ 10 ]
 Chapter 1

If we keep adding more and more experiments and therefore more and more variables,
we can write a very big and complex joint probability distribution. For example, I
could be interested in the probability that it will rain tomorrow, that the stock market
will rise and that there will be a traffic jam on the highway that I take to go to work.
It's a complex one but not unrealistic. I'm almost sure that the stock market and the
weather are really not dependent. However, the traffic condition and the weather
are seriously connected. I would like to write the distribution P(W, M, T)—weather,
market, traffic—but it seems to be overly complex. In fact, it is not and this is what we
will see throughout this book.

A probabilistic graphical model is a joint probability distribution. And nothing else.

One last and very important notion regarding joint probability distributions is
marginalization. When you have a probability distribution over several random
variables, that is a joint probability distribution, you may want to eliminate some of
the variables from this distribution to have a distribution on fewer variables. This
operation is very important. The marginal distribution p(X) of a joint distribution
p(X, Y) is obtained by the following operation:

p ( X ) = ∑ y p ( X , Y ) where we sum the probabilities over all the possible values of y.

By doing so, you can eliminate Y from P(X, Y). As an exercise, I'll let you think about
the link between this and the probability of two disjoint events that we saw earlier.

For the math-orientated readers, when Y is a continuous variable, the

marginalization can be written as p ( X ) = ∫ y p ( X , y ) dy .

This operation is extremely important and hard to compute for probabilistic

graphical models and most if not all the algorithms for PGM try to propose an
efficient solution to this problem. Thanks to these algorithms, we can do complex yet
efficient models on many variables with real-world data.

Bayes' rule
Let's continue our exploration of the basic concepts we need to play with
probabilistic graphical models. We saw the notion of marginalization, which
is important because, when you have a complex model, you may want to
extract information about one or a few variables of interest. And this is when
marginalization is used.

But the two most important concepts are conditional probability and Bayes' rule.

[ 11 ]
Probabilistic Reasoning

A conditional probability is the probability of an event conditioned on the

knowledge of another event. Obviously, the two events must be somehow
dependent otherwise knowing one will not change anything for the other:

• What's the probability of rain tomorrow? And what's the probability of a

traffic jam in the streets?
• Knowing it's going to rain tomorrow, what is now the probability of a traffic
jam? Presumably higher than if you knew nothing.

This is a conditional probability. In more formal terms, we can write the following
formula:

p ( X ,Y ) P ( X ,Y )
p( X |Y ) = and P (Y | X ) =
p (Y ) P( X )
From these two equations we can easily deduce the Bayes formula:

P ( Y | X ) .P ( X )
P( X |Y ) =
P (Y )

This formula is the most important and it helps invert probabilistic relationships.
This is the chef d'oeuvre of Laplace's career and one of the most important formulas in
modern science. Yet it is very simple.

In this formula, we call P(X | Y) the posterior distribution of X given Y. Therefore,

we also call P(X) the prior distribution. We also call P(Y | X) the likelihood and
finally P(Y) is the normalization factor.

The normalization factor needs a bit of explanation and development here. Recall
that P ( X , Y ) = P (Y | X ) P ( X ) . And also, we saw that P (Y ) = ∑ x P ( X , Y ) , an operation
we called marginalization, whose goal was to eliminate (or marginalize out) a
variable from a joint probability distribution.

So from there, we can write P (Y ) = ∑ x P ( X , Y ) = ∑ x P (Y | X ) P ( X ) .

Thanks to this magic bit of simple algebra, we can rewrite the Bayes' formula in its
general form and also the most convenient one:

P ( Y | X ) .P ( X )
P( X |Y ) =
∑ x P (Y | X ) P ( X )

[ 12 ]
 Another Random Document on
Scribd Without Any Related Topics
excavation consists of earth or gravel, it ought to be spread over the
whole mossy surface, whether the field be morass or drier hill-peat:
this would be useful in consolidating it, and in preventing too great
exhaustion of moisture in severe droughts, from which vegetation in
moss-soil suffers so much. Even though planting were not intended,
this fluting and top-dressing would facilitate the raising of the
gramineæ. These ditches, when the ground is not too stoney, or too
moist, or containing roots, might be scooped out, excepting a little
help at the bottom, by means of a scoop-sledge, or levelling box,
worked by a man and two horses, the surface being always loosened
by the common plough: one of these will remove earth as fast as
twenty men with wheelbarrows.
ON BENDING AND KNEEING LARCH.
We cannot too forcibly inculcate the urgent necessity of attending
to the bending of the larch: for our country’s interest, we almost
regret we cannot compel it. In all larch plantations, in proper {91}
soil, not too far advanced, and in all that may hereafter be planted,
a proportion of those intended to remain as standards should be
bended. The most proper time for this would perhaps be May or
June, before the top-growth commences, or has advanced far; the
best size is from three feet high and upwards. The plants should be
bent the first season to an angle of from 40° to 60° with the
horizon, and the next brought down to from 10° to 60°, according to
the size of the plant, or the curve required,—the smallest plants to
the lowest angle.
From experience we find that the roots of larch form the best of
all knees; they, however, might be much improved by culture22,
although it does not {92} seem as yet to have been attempted or
thought of. To form the roots properly into knees, should the plants
be pretty large, the planter ought to select those plants which have
four main roots springing out nearly at right angles, the regularity of
which he may improve a little by pruning, and plant them out as
standards in the thinnest dryest soil suited for larch, carefully
spreading the roots to equal distances and in a horizontal position.
To promote the regular square diverging of these four roots, he
should dig narrow ruts about a foot deep and three feet long out
from the point of each root, and fill them in with the richest of the
neighbouring turf along with a little manure. When the plants are
small, and the roots only a tuft of fibres, he should dig two narrow
ruts about eight feet long crossing each other at the middle at right
angles, fill these as above, and put in the plant at the crossing: the
rich mould of the rotted turf and its softness from being dug, will
cause the plant to throw out its roots in the form of a cross along
the trenches. When the plants have reached five or six feet in
height, the earth may be removed a little from the root, and, if more
than one stout root leader have run out into any of the four
trenches, or if any have entered the unstirred earth, they ought all
to be cut excepting one, the stoutest {93} and most regular in each
trench. In a few years afterwards, when the plants have acquired
some strength, the earth should be removed gradually, baring the
roots to from two to five feet distance from the stool, or as far as
the main spurs have kept straight, cutting off any side-shoots within
this distance, should it be found that such late root-pruning does not
induce rot. This process of baring the roots will scarcely injure the
growth of the trees, as the roots draw the necessary pabulum from a
considerable distance, nor, if done carefully, will it endanger their
upsetting; and the roots, from exposure to the air, will swell to
extraordinary size23, so as to render them, ere long, the firmest
rooted trees in the wood. The labour of this not amounting to the
value of sixpence each, will be counterbalanced thrice {94} over by
the ease of grubbing the roots for knees; and the whole brought to
the shipwright will produce more than double the price that the
straight tree alone would have done.
 The forester should also examine and probe the roots of his
growing larch, even those of considerable size, in sound ground; and
when several strong horizontal spurs, not exceeding four, are
discovered nearly straight, and from two to five feet long, he ought
to bare these roots to that distance, that they may swell, carefully
pruning away any small side-roots, and reserve these plants as
valuable store , taking good heed that no cart-wheel in passing, or
feet of large quadruped, wound the bared roots. In exposed
situations the earth may be gradually removed from the roots.
The rot in larch taking place in the part appropriate to knees, the
forester cannot be too wary in selecting the situations where there is
no risk of its attack, for planting those destined for this purpose. It is
also desirable, if possible, to have the knee timber in ground free of
stones or gravel, as the grubbing in stoney ground is expensive, and
the roots often embrace stones which, by the future swelling of the
bulb, are completely imbedded and shut up in the wood, particularly
in those places between the spurs {95} where the saw section has to
divide them for knees. Were the roots carefully bared at an early
period, it would tend to prevent the gravel from becoming imbedded
in the bulb. Nothing can be more annoying to the shipwright, when
he has bestowed his money, ingenuity, and labour, upon an unwieldy
root, and brought his knees into figure at the cost of the destruction
of his tools by the enveloped gravel, to discover stains of incipient
rot which renders it lumber.
This plan of baring the roots might be extended to oak trees for
knees, baring and pruning about a foot out from the bulb annually.
By exposure to the air, the timber of the root would mature and
become red wood of sufficient durability. When covered with earth,
the root of the oak remains white or sap wood, and soon decays
after being dug up, the matured wood of the stem scarcely
extending at all underneath the surface of the ground. The roots of
the pine tribe are the reverse of this, at least the bulb and the spurs
near it, are the best matured, reddest, toughest, most resinous, part
of the tree. It is probably unnecessary to observe, that it would be
folly to remove the earth from the bulb of trees in situations where
water would stand for any length of time in the excavation. {96}
Larch knees are possessed of such strength and durability, and are
of such adaptation by their figure and toughness, that were a
sufficient quantity in the market, and their qualities generally known,
we believe that none else would be used for vessels of any
description of timber—even for our war-navy of oak. In America,
where it is difficult to procure good oak knees in their close forest, it
is customary to use them of spruce roots even for their finest
vessels. The knees of vessels have a number of strong bolts,
generally of iron, passing through them to secure the beam-ends to
the sides of the ship. Larch knees are the more suited for this, as
they do not split in the driving of the bolts, and contain a resinous
gum which prevents the oxidation of the iron.
As the larch, unlike the oak, affords few or no crooks naturally,
excepting knees, the artificial formation of larch crooks is of the
utmost consequence to the interest of the holders of larch
plantations now growing. In order to obtain a good market for their
straight timber, it is absolutely necessary to have a supply of crooks
ready as soon as possible to work the straight up. This would
increase the demand, and thence enhance the price of the straight
more than any one not belonging to the craft could {97} believe. In
good soil many of the crooks would be of sufficient size in twenty
years to begin the supply, if properly thinned out. In a forest of larch
containing many thousand loads, and which had been untouched by
any builder, we have seen the greatest difficulty in procuring crooks
for one small brig. It is only on very steep ground, and where the
tree has been a little upset after planting, that any good crooks are
found. From the rather greater diameter required of larch timbers,
and also from the nature of the fibre of the wood, we should
suppose that steam bending of larch timbers would scarcely be
followed, even as a dernier ressort .
Larch, from its great lateral toughness, particularly the root, and
from its lightness, seems better adapted for the construction of shot-
proof vessels than any other timber; and opposed end-way to shot
in a layer, arch fashion, several feet deep around a vessel, would
sustain more battering than any other subject we are acquainted
with, metal excepted. Were the part above water of a strong steam-
vessel, having the paddles under cover, a section of a spheroid or
half egg cut longitudinally, and covered all around with the root cuts
of larch five or six feet deep with the hewn down bulb, external; well
supported {98} inside, having nothing exposed outside of this arch,
and only a few small holes for ventilators and eyes; there is no shot
in present naval use that would have much impression upon it. Had
such a vessel a great impelling power, and a very strong iron
cutwater, or short beak wedge-shaped (in manner of the old Grecian
galleys), projecting before the vessel under water, well supported
within by beams radiating back in all directions, she might be
wrought to split and sink a fleet of men-of-war lying becalmed, in a
few hours. This could be done by running successively against each,
midships, and on percussion immediately backing the engine, at
same time spouting forth missiles, hot water, or sulphuric acid from
the bow to obstruct boarding; but even though the external arch
were covered with assailants like a swarm of bees, they would be
harmless, or could be easily displaced. To prevent combustion by red
hot shot, the larch blocks, after drying, might have their pores filled
by pressure with alkali. However, the employment of bomb-cannon
about to be introduced in naval warfare, throwing explosive shot,
regulated with just sufficient force to penetrate without passing
through the side of the opposed vessel, will render any other than
metallic defensive cover ineffectual; but {99} this circumstance will,
at the same time, completely revolutionize sea affairs, laying on the
shelf our huge men-of-war, whose place will be occupied with
numerous bomb-cannon boats, whose small size will render them
difficult to be hit, and from which one single explosive shot taking
effect low down in the large exposed side of a three decker will tear
open a breach sufficient to sink her almost instantly. For the
construction of these boats, larch, especially were a proportion bent,
would be extremely suitable, and thence larch will probably, ere
long, become our naval stay.
Larch has been used in the building-yards of the Tay for 20 years
back; and there is now afloat several thousand tons of shipping
constructed of it. The Athole Frigate built of it nearly 12 years ago,
the Larch, a fine brig built by the Duke of Athole several years
earlier, and many other vessels built more recently, prove that larch
is as valuable for naval purposes as the most sanguine had
anticipated. The first instance we have heard of British larch being
used in this manner, was in a sloop repaired with it about 22 years
back. The person to whom it had belonged, and who had sailed it
himself, stated to us immediately after its loss, that this sloop had
been built of oak about 36 years before; that at 18 years {100} old
her upper timbers were so much decayed as to require renewal,
which was done with larch; that 18 years after this repair this sloop
went to pieces on the remains of the pier of Methel, Fifeshire, and
the top timbers and second foot-hooks of larch were washed ashore
as tough and sound as when first put into the vessel, not one spot of
decay appearing, they having assumed the blue dark colour which
some timber acquires in moist situations, when it may be stiled
cured ; being either no longer liable to the putrid change constituting
dry rot, or which forms timber into a proper soil for the growth of
dry rot; or, from this blueness caused by the union of the tannin with
iron acting as a poison on vegetation: this blueness, resulting from
some alteration in the balance of affinities, occurs chiefly in timber
containing much of the tannin principle, in which larch abounds. The
owner of a larch brig who had employed her for several years on
tropical voyages, also assures us that the timber will wear well in
any climate, and that he would prefer larch to any other kind of
wood, especially for small vessels; he also states that the deck of
this brig, composed of larch plank, stood the tropical heat well, and
that it did not warp or shrink as was apprehended.
From the softness of the fibre and want of {101} density of the
larch, we would not deem it suitable for planking vessels beyond the
size of ordinary merchantmen, say 500 tons, as in the straining of
very large vessels, when the greatest force comes upon the outward
skin, the fabric of the wood might crush before it, along the edge of
the plank, and throw (chew) the oakum. In ordinary sized vessels,
however, larch plank retains the oakum better than oak, from
greater lateral elasticity. For the purpose of timbers, if root-cuts24,
and properly bent, we would think larch suitable to the largest class
of vessels; as, though light, it is tough and quite free from knot,
crack, or cross-grain, which is so common in oak, and which
occasions dense old oak in large masses to give way at once, before
a shock or strain, the hardness and unyielding nature of the fibre
concentrating the whole dirupting impetus to one point. Larch may
also be advantageously employed in the ceiling or inside skin of the
part of war vessels above water: shot bores it, comparatively, like an
auger,—thence the structure will endure longer under fire, and life
be much economized.
In all places where larch has become known, it has completely
superseded other timber for {102} clinker-built boats, surpassing all
others in strength, lightness, and durability. For this purpose, young
trees of about 9 inches diameter, in root-cuts from 10 to 20 feet in
length, with a gentle bend at one end, such as the larch often
receives from the south-west wind, are the most suitable. The log
should be kept in the bark till used, and in dry weather the boards
put upon the boat’s side within two or three days from being sawn
out, as no timber we are acquainted with parts sooner with its
moisture than larch; and the boards do not work or bend pleasantly
when dry. When dried, the thin larch board is at once strong, tough,
durable, and extremely light. The tough strength, almost equalling
leather, is owing to the woven or netted structure of the fibre of the
wood, entirely different from the pine, whose reedy structure runs
parallel with very slight connecting or diverging fibres. It is very
difficult to split larch even by wedges.
For rural purposes generally, larch is incomparably the best
adapted timber, especially for rail or fence, or out-door fabric
exposed to wind and weather. It is also getting into use for
implements of husbandry, such as harrows, ploughs, and carts. We
have seen a larch upright paling, the timber of which, with the
exception of the large charred posts, {103} had only been eight years
in growing, standing a good fence, sixteen years old, decked out by
moss and lichen in all the hoary garniture of time.
In the construction of buildings, larch is valuable only for the
grosser parts, as beams, lintels, joists, couples. For the finer boarded
part, it is so much disposed to warp, and so difficult to be worked,
as generally to preclude use. It is, however, asserted that if larch be
seasoned by standing two years with the bark stripped from the bole
before being cut down, that the timber becomes manageable for
finer house work.
Although larch timber be extremely durable in exposed situation,
yet it yields to the depredations of insects fully as soon as any pine
timber in close houses. We have proof of it in house-furniture about
50 years old, but it is considerably moth-eaten by apparently a
smaller insect than common. Larch stools also disappear in forests
sooner than the stools of Scots fir, being eaten by a species of
beetle; and the sea-worm devours larch in preference to almost any
other wood.
We have looked over some experiments conducted at Woolwich, in
trial of the comparative strength of larch and other fir timber, where
the larch is stated inferior to Riga and Dantzic fir, Pitch pine, {104}
and even Yellow pine. Larch, in the districts of Scotland where it is
grown and much in use, is universally allowed to be considerably
stronger than other fir; and the sawyers of it have one-fourth more
pay per stated measure. We, ourselves, have had considerable
experience of the strength of larch applied to many purposes, and
have found it in general much superior in strength to other fir. We
have known a crooked topmast of this timber, to which the sailors
bore a grudge, defy their utmost ingenuity to get carried away. We
once had four double horse-carts, made (excepting the wheels) of
peeled young larch of rather slow growth, for the carriage of large
stones; these, by mistake, were made very slight, so light, that,
without the wheels, a man could have carried one of them away.
When we saw the first loading of stones nearly a ton weight each,
two in each cart, and the timber yielding and creaking like a willow-
basket, we did not expect they would have supported the weight
and jostlings of a rugged road many yards; yet they withstood this
coarse employment for a long time. The timber of larch near the top
of the tree is, however, very inferior and deficient in toughness; and
it is not improbable that the experiments above alluded to at
Woolwich had been made with larch {105} timber deficient in strength
from being a top. White larch has comparatively smaller and more
numerous branches than any other of the Coniferæ; consequently
the timber is freer of large knots, and has more equable strength, as
well in small spars as when large and cut out into joists and beams,
provided the timber be not too far up the tree. Larch, however,
compared with pines and firs, has the timber much stronger when
young, and several inches or below a foot in diameter, than when old
and large : this may partly be owing to its deficiency in resinous
deposit.
 NOTES TO PART II.
10. We have often preferred the terms kind, breed, family, individual, to genus,
species, variety, subvariety, as the former seem less definite. Were nature true to
the latter classification as employed by botanists, it would be convenient.
11. In those we observed, we considered this last circumstance had a considerable
share as a predisposing cause of the attack of the worm. Forests of Pinus
sylvestris are sometimes destroyed by insects under the bark, in cases where it is
difficult to decide whether external circumstances, such as a dry warm season, has
been promotive of the increase of the insect itself, or has induced some disorder in
the plant, rendering the juices more suitable aliment to the worm.
12. Some nautical or technical terms have unavoidably crept into this work; we
shall not presume to think any explanation necessary: Britannia would blush
jusqu’au blanc des yeux , to the tips of the fingers and toes, did she think it were
doubted that any of her sons, not doomed to unceasing mechanical labour, were
unacquainted with these.
13. It is termed by our professors Salix fragilis , or Crack Willow, from the small
branches breaking easily at the junction of the annual growth—or, perhaps, Crack
Willow, from the branches breaking with considerable report; or from the wood,
while burning, frequently detonating or crackling, from the expansion of some
aërial fluid within the fibres. Though named by their sapience fragilis , it is not
weaker than other large growing willows, but stronger and denser; and, being
harder in the small branches, they do not bend, but break when their bark and
alburnum is driest, in winter. The timber is superior to that of Salix alba , or of any
other large growing willow we are acquainted with, and is sufficiently pliant and
tough.
14. Red Canadian pine is generally termed Pinus resinosa ; but as it is not so
resinous as several other kinds, we consider Pinus rubra (rubra from the colour of
stem and also of timber), which is sometimes used, more suitable. The pitch pine
of the American United States should be Pinus resinosa .
15. We think that in mankind the variations of the children of the same parents do
not soften entirely—there would seem to be certain types or nuclei both of
appearance and temperament around which external and internal character
vibrates.
16. The Canadian red pine resembles P. sylvestris or Norway pine so much, that it
is usually styled Norway pine by the settlers: Though different, it is so nearly allied
to P. sylvestris, that we consider the number of sap-growths may be referred to
the climate and soil, and not to the kind,—that is, that, were it grown in Britain, if
it did not at first, it would in the course of time come to have fewer sap-growths.
17. Our common larch, like almost every other kind of tree, consists of numberless
varieties, which differ considerably in quickness of growth, ultimate size, and value
of timber. This subject has been much neglected. We are, however, on the eve of
great improvements in arboriculture; the qualities and habits of varieties are just
beginning to be studied. It is also found that the uniformity in each kind of wild
growing plants called species , may be broken down by art or culture, and that
when once a breach is made, there is almost no limit to disorder; the mele that
ensues being nearly incapable of reduction.
18. There is yet no sufficient data for the term alpine plant, but with reference to
latitude. The influence on vegetables, arising from rarefaction and diminution of
pressure of atmosphere, from difference of stimulus of solar ray—when the entire
ray of light, heat, and chemical power, though less intense, is radiated fresh, and
not much broken or modified by refraction and reflection, and heat communicated
more in proportion by radiation than by contact of heated air; or from difference of
electric or galvanic or other meteoric impression connected with altitude or ranges
of mountains, or with primary rocks or more upright strata, has not been made
the subject of research, at least has not been sufficiently investigated by any
naturalist.
19. When water is stationary, either in the pores of the soil or by itself, if the
temperature be not very low, a slight putrefaction generally commences, aided by
the dead vegetable or animal matter contained in the soil or the water; and it is
only the more robust aquatic vegetables whose juices are not corrupted, from
their roots being soaked in this tainted fluid. It would appear, too, that the
aqueous part of the atmosphere is also susceptible of the same putrid changes,
although in general the putrescency may have commenced before the
evaporation. This condition of the aqueous part of the atmosphere is a disposing
cause to blight or mildew in vegetables, and remittent, intermittent, and putrid
fevers in man. Mill-ponds are notorious both for mildew and agues.
20. We have had no experience of larch, excepting very young, growing on chalk
and its affinities. We are told there are a few instances where larch has reached 50
years in these calcareous soils, some distance south of London. This merits
attention.
21. “Oh! the bonny blooming heather.”—“Man has spoken evil things of the sun, of
love, and of life.”
22. As we held this plan of forming larch knees, and of bending larch, of
considerable importance, we some time ago presented it in manuscript, along with
some other matter, to the Highland Society of Scotland. Tiring, however, of the
delay of examination, perhaps unavoidable in their official departments, and from
some improvements occurring to us during the delay, we requested it back. We
now present it under this more convenient form to the Society, and hope they will
find the examination or perusal of it printed, not quite so impracticable as when in
manuscript. It will afford us pleasure to know that this useful Society approves,
and that the members who have opportunity are setting about following our
directions. We especially recommend to them to probe the roots of their growing
larch, and to lay bare those fitted for knees.
23. The landlord agriculturist is sufficiently aware of the influence of the baring the
upper part of the root of turnip, while the plant is young, in extending the future
growth of the bulb, and that a dry situation gives most root in proportion to stem.
These are general laws in vegetation. There are few observers who have not
remarked the very large size which roots have attained when the trees have
originally been planted on dikes, and the dike earth removed, leaving the roots
bare. Should any person examine the very great difference of thickness between
the upper and lower part, from the heart of a root near the bulb, he will at once
discover the influence of exposure to the air and freeness from pressure in
promoting the swelling.
24. As you ascend the tree the timber deteriorates greatly.
 PART III.
MISCELLANEOUS MATTER CONNECTED
WITH NAVAL TIMBER.

NURSERIES.
Much of the luxuriance and size of timber depending upon the
particular variety of the species, upon the treatment of the seed
before sowing, and upon the treatment of the young plant, and as
this fundamental subject is neither much attended to nor generally
understood, we shall take it up ab initio .
The consequences are now being developed of our deplorable
ignorance of, or inattention to, one of the most evident traits of
natural history, that vegetables as well as animals are generally
liable to an almost unlimited diversification, regulated by climate,
soil, nourishment, and new commixture of already formed varieties.
In those with which man is most intimate, and where his agency in
throwing them from their natural locality and dispositions has
brought out this power of diversification in stronger shades, it has
been forced upon his notice, as in man himself, in the dog, horse,
cow, sheep, poultry,—in the apple, {107} pear, plum, gooseberry,
potato, pea, which sport in infinite varieties, differing considerably in
size, colour, taste, firmness of texture, period of growth, almost in
every recognisable quality. In all these kinds man is influencial in
preventing deterioration, by careful selection of the largest or most
valuable as breeders; but in timber trees the opposite course has
been pursued. The large growing varieties being so long of coming
to produce seed, that many plantations are cut down before they
reach this maturity, the small growing and weakly varieties, known
by early and extreme seeding, have been continually selected as
reproductive stock, from the ease and conveniency with which their
seed could be procured; and the husks of several kinds of these
invariably kiln-dried25, in order that the seeds might be the more
easily extracted! May we, then, wonder that our plantations are
occupied by a sickly short-lived puny race, incapable of supporting
existence in situations where their own kind had formerly flourished
—particularly evinced in the genus {108} Pinus, more particularly in
the species Scots fir; so much inferior to those of Nature’s own
rearing, where only the stronger, more hardy, soil-suited varieties
can struggle forward to maturity and reproduction?
We say that the rural economist should pay as much regard to the
breed or particular variety of his forest trees, as he does to that of
his live stock of horses, cows, and sheep. That nurserymen should
attest the variety of their timber plants, sowing no seeds but those
gathered from the largest, most healthy, and luxuriant growing
trees, abstaining from the seed of the prematurely productive, and
also from that of the very aged and over-mature; as they, from
animal analogy, may be expected to give an infirm progeny, subject
to premature decay.
As, from many facts, a considerable influence is known to result in
several vegetables from drying severely the seeds from whence they
had sprung26,—from exposure of these seeds to the sun and air,—
from long keeping, or from injury by mould or {109} impure air, which
all tend to shorten the life of the resulting individual, to accelerate
the period of its seeding, and to increase its reproductiveness; the
nurseryman should pay the utmost attention to the seeds he makes
use of, procuring them as recent as possible, and preserving them in
well-aired lofts, or under sheds, and also retaining them in the husks
till the time of sowing: the superior germinating power of the seed
thus treated will repay this attention.
 From facts we are also assured, that, in some hard wood kinds,
and also in the Coniferæ, the hanging of the growth of the young
plant, the spindling up in the seed-bed, or injudicious deterring
treatment afterwards, have a tendency to injure the constitution of
the individual, inducing premature seeding, and diminutive old age;
and also, that when plants, especially of some size, of these kinds of
trees have their roots much broken, the secondary or new roots
often partake something of the nature of the infirm runners, which,
in most kinds of trees, are thrown out by layers,—the resulting tree,
as in the case of those from layers in fruit trees being dwarfish,
sooner exhausting itself by reproduction, and sooner decaying. For
distinctness, we shall recapitulate: {110}
That the seed be from the largest, hardiest variety of tree in
luxuriant growth.
That the seed be recent, and carefully preserved in husk till
sowing, and extracted from the husk or cone without artificial
drying.
That the nursery be in an open, rather exposed situation,—most
eligible without shelter either of tree, hedge or wall, of rather light
dry soil of ordinary quality, of dry climate, and, in preference, soil
naturally good to that made so by high manuring.
That the plants be not too close, nor remain too long in the seed-
bed; that they be extricated without much fracture of root, and be
replanted in wide rows, with good space between the plants in the
row, keeping the roots as superficially extended as they will thrive,
and without doubling the main root up to the surface of the ground.
That the plant receive no pruning, excepting in the case of more
than one leader appearing, or feeder unproportionally extended; and
no root-section, in order to retard its growth, or increase the number
of root-fibres; and that its ultimate removal be accomplished without
much fracture of root or branch.
 By exposed situation of nursery, ordinary quality of soil, and much
room in the seed-bed and rows, we {111} shall have plants with firm
fibre and hardy constitution, with thick juicy bark, thick stem at the
surface of ground, and numerous feeders all the way down the
stem. Roots are most easily extricated from light soil, and with least
fracture. They are large in proportion to stem in dry soil and climate,
and when they are situated near the surface of the ground.—A
healthy growing plant, of firm fibre, large root, and sturdy short
stem of one leader and numerous feeders, is the great desideratum:
a large root is the more desirable, as a considerable part of it is
generally broken off in transplanting, rendering it disproportioned to
the top, which, in consequence, either languishes, or receives
deterring cropping.
We consider, that a tree grows more luxuriantly, acquires larger
size, and is much longer of reaching senility, when it is furnished
with several large roots, say one or two to each of the cardinal
points, extending horizontally out with bold leaders, than when
numerous small rootlets diverge in all directions from the bulb, as is
the case in some kinds when much fracture of root takes place from
frequent removals, or, when the nursery is of moist or mossy soil,
the plants being removed when of considerable size. We have cut
down old stunted hard wood trees having extremely numerous
crowded roots, all {112} engrafted into a matted net throughout the
soil near the bulb, and without any strong extended leaders. We
attributed this crowded rooting to the plants having been of
considerable size when put in, and losing their natural leaders; the
situation, an avenue exposed to cattle, went to confirm the
probability that the defect of the rooting had been owing to the
largeness of the plants.
When a tree is supplied by numerous, consequently small and not
wide-extending roots, as the tree acquires size, the wide spreading
branches and leafy top shed off the rain and dews from the space
occupied by these roots, very few of them extending beyond this
shade; at the same time, this narrow space becomes soon
exhausted of the more particular pabulum necessary to the kind of
plant, the exhaustion being accelerated by the dryness. This dryness
and exhaustion of the soil very soon show their effects aloft; the
living hark of the tree becomes covered from its connexion with the
air, and constricted by a thick hard dead crust, which, with the
consequent very thin alburnum affording an inefficient
communication between the supply and demand, react to impair the
general vigour, and particularly to impede the descent of the proper
sap necessary to the enlargement and further extension of the roots.
The buds {113} not receiving sufficient supply of root-moisture,
instead of pressing on to new formation of wood, only find enough
to burgeon out into flower-buds, which the following season drain
the tree by reproduction; this fruit-bearing alternates with periods of
exhaustion, when the buds have not even supply sufficient to swell
into the embryo of flower and seed, but extend only into a few
leaves; and sometimes, in the event of a benign season, the buds
may throw out a small extension of new shoots. The tree progresses
very slowly in thickness of bole all this time, and generally soon falls
a prey to disease. On the other hand, when the tree has its naturally
fine large roots preserved, and is situated in open forest, and mixed
with other kinds, these large roots diverging widely from the tree
and each other, have a much larger less-sought space to forage in;
and the tree enjoying a long period of luxuriant growth before it fall
much into seed-bearing, acquires strength of constitution to thrive
and increase for ages under this drain.
We are satisfied that cutting or fracture of the root-leaders,
especially near the bulb, when they have acquired some size, is
injurious to the extension and longevity of the tree, in pines and
most kinds of hard wood; and that branch-pruning, as generally
practised, is not less pernicious, first, by the {114} derangement
which the plant receives, from the regular connexion between the
rootlets and their affiliated twigs and leaves being destroyed by the
section, and afterwards from the distance between the
manufacturing parts, the leaves and the sources of supply in the
ground being unnaturally extended, especially when the stem is
long, slender, and much denuded.
Although we consider severe root fracture at planting pernicious to
some hard wood and resinous trees, yet there are kinds to which it
is advantageous. All plants which grow freely by cuttings, strike
better to have the roots pruned in near to the bulb. Many kinds of
seedling-plants also strike sooner, and throw out stronger new root-
leaders, when the long straggling fibres are cut in a little, similar to
the branches above, which, when over-numerous and slender, throw
out more vigorous shoots by being cropped at planting.
 PLANTING.
In regard to planting, soils divide into the dry and the moist ; the
former require to have the plants put in as soon as possible after the
leaves drop off—at any rate, not to allow February to pass without
completing the planting; excepting evergreens, {115} which should
not be delayed beyond the middle of April. In dry soils, if the
expense be not limited to a very low rate, pit-planting should be
adopted, and the pits are better to be dug some months previous, in
order that the earth may be aërated, and the turf partly rotted. The
moist soils may be divided into those which are much disposed to
throw the plant from the frosts and thaws, and those which are not;
the former consisting of moory, soft, or spongy earth, upon a
retentive subsoil; the latter, of the firmer, more equable loams, clays,
and tills. Unless the plants are large, they should always be slitted
into the former soil, and the work performed as soon as the ground
becomes sadded in spring—as, though the lateness of planting
should preclude throwing of pitted plants the first season, they will
often be thrown the ensuing winter. When plants are very small,
they may be put into the latter, by slitting; but if middle-sized, or
large, they are better pitted. It is of the greatest importance to these
moist soils, to have very deep, open27 drains executed previous to
planting, cutting off all the springs at their sources, and, if possible,
drying the subsoil to such a degree that water will not stand in the
pits. Should this be {116} accomplished, it is highly advantageous to
dig the pits in time for the excavated clay to have its cohesion
broken by frost: the planting should afterwards be performed exactly
at the time when this frosted mould is sufficiently dry, and no more,
to shake conveniently in among the fibres of the roots, and not to
knead into mortar, by the necessary pressing of the feet. After this
pressure, a little of the tenderest of the soil should be spread loose
over the surface, to exclude drought. Should this dryness of subsoil
not be effected, the pits must be dug in spring, at the time the clay
is most friable; that is, between the moist and dry; and the plants
put in immediately, breaking the clay as fine as possible, and closing
it well around the roots. It is better to delay planting even till May,
than to perform it too wet. When planting is delayed late in spring,
the plants should be kept shoughed in the coldest situation that can
be found, at the top of a hill exposed to the north, or in some cold,
damp, back-lying place. Care should also be taken not to expose
them much while planting, as they, especially if the buds be
bursting, very soon wither when root and stem are both exposed to
the sun and dry air. When late planted, they ought always to be
dipped as far up as the branches in a puddle of clay and water: {117}
should they be dipped over head in the puddle, it will not injure
them.
What is of most importance to the success of planting, is to have
the soil put very closely in contact with all the root-fibre, and these
fibres in due natural separation, with a little tender mould on the
surface;—not to have water stagnating around the root, at any rate
during the first spring;—to have the planting done in time, to receive
a good sadding by rain before the spring droughts commence;—to
prevent rank weeds, furze, &c. from smothering the young plants;—
and to exclude or destroy all bestial, as cattle, sheep, rabbits, hares,
mice, &c. In keeping the latter in check, a few families of foxes are
very efficient.
 FURTHER OBSERVATIONS ON PRUNING.
Every forester is aware, that when feeders are pruned off, they
should be cut away as close as possible to, and without tearing the
hole. To perform this without danger of injury to the tree, when
feeders of considerable size are to be removed, the branch should
first be sawn over at about one foot beyond the intended section,
and a second section then performed at the proper place. This {118}
requires a little more time, but not nearly so much as an
inexperienced person would suppose, as the section a foot out is
made very quickly, and the pruner generally takes as much time to
reach the branch as to cut it off. The neatness and advantage of this
method will be acknowledged by those who have seen it practised,
to compensate for the longer time it requires.
We find the saw, shears, and knife, the best instruments for
pruning; in some cases of difficult approach, the long-handed
pruning-iron may be resorted to. When the lopping is performed by
a percussion tool, the wood and bark at the section is often
shattered by the blow, and thence is less likely to cicatrize soundly;
and even when executed in the best manner, the surface of the
section is smooth and hard, consequently a good conductor of heat,
dries much, and thence shrinks and cracks near the centre of the
cut, opening a deep crevice, into which the rain penetrates, and
often rots deep into the stem. When the section is made by the saw,
a slight fibrous clothing is left upon the place, which in some
measure protects the ends of the cut tubes from the frost and drying
air, and excludes the heat; in consequence the wood at the section
does not lose its vitality so far inward, and is not so liable to shrink
{119} and crack in the centre and receive rain. The section can also
generally be made much neater and closer by the saw than by any
other instrument. The common erroneous belief, that a section by a
sharp-edged instrument is less injurious than by the saw, is merely
hypothetical, from wide analogy from animals. The pernicious
influence on the whole individual, received and transmitted by the
nerves from mangled section of animal fibre, is probably entirely
awanting in vegetables; the whole process of life and of cicatrization
is also totally different.
The forester should also be very wary in cutting off a considerable
branch, whose section would incline upwards, as such a section,
when it has received a circle of new bark and wood, forms a cup
which receives and contains rain water, which quickly corrupts the
bottom of the cup, and often rots the centre of the tree down to the
ground. It is better to crop such a branch several feet from the main
stem, close by some small feeder, unless the branch be dead. In
pruning, every considerable section should be as near as possible at
right angles with the horizon, or rather inclining inward below. Of
naval timber, the beech is by far the most likely to take rot by being
pruned, and should never have a large limb cut off, as the divided
fibres generally die {120} downward a number of feet below the
section, and soon afterward decay, leaving a hole in the bole.
As nothing retards the growth of trees more than full flowering
and seeding, if pruning diminish this flowering and seeding, so that
the gain from the prevention of this exhaustion more than
counterbalances the loss of the pruned-off part, the pruning will of
course accelerate the growth of the tree; but the removal of lower
branches, although in the first place promotive of growing buds and
extension of the top, in a year or two longer only tends to throw the
tree more into flowering and seeding. The rich dryness, or want of
fluidity of the juices which occasions flower-buds, is also induced by
hot, dry atmosphere, and short supply of moisture from the roots
during the preceding summer, both of which disposing causes are
increased by a long naked stem. When the proportion of the part
above ground of a tree to the roots is diminished, growing buds
result, at least to a certain extent; yet it would be very difficult to
practise a proper system of pruning on this principle, as the
consequent lengthened stem is, in the end, promotive of flower-
buds, especially in dry seasons, and the loss of feeders might greatly
counterbalance the gain from not flowering, did a succession of wet
cold seasons follow. {121}
The season when pruning should be performed, is something
dependent upon the kinds, whether they bleed when pruned in early
spring or do not. Almost any convenient time will suit for pruning the
latter, but we rather prefer March, April, May, June, or autumn after
the leaf has fallen. The former, sycamore, maple, birch, &c. ought
either to be pruned in autumn, or after the buds are beginning to
break in spring, as they bleed and suffer considerable exhaustion
when pruned in the latter part of winter or early spring. From some
facts, we consider that pruning in winter, especially in severe
weather, gives a check to the vigour of the tree; others agree with
this.
 OBSERVATIONS ON TIMBER.
The quantity of measurable wood of the various timber trees
which a certain extent of adapted ground will carry, when come to
full maturity, or when they may be most profitably felled, and the
quantity that may be thinned out during the maturing, with the time
requisite to bring to value, with the relative selling price per foot,
and also whether the greatest quantity of timber can be grown of
one kind or mixed, are questions of more importance than might be
judged, from the attention paid to the subject. Of our common
timber trees, Scots fir, silver fir, and spruce, larch, pinaster, black
Italian poplar, Salix alba, commonly called Huntingdon willow, red-
wood willow, beech, Spanish chestnut, ash, plane, elm, birch, oak,
are here ranked nearly in the order of quantity of measure which
adapted ground in this country will produce or support; that is, that
an acre of close Scots fir trees, of whatever age, will admeasure
more timber than an acre covered with any other tree of the same
size; and a close acre of oaks less. A little further south, in the
temperate zone, the large-leaved deciduous trees, particularly the
{123} elms, acquire thicker and longer stem, in closer order, in a
given time. In this country, in rich warm situations, this is visible in
some degree, both as regards quantity of timber and quickness of
growth, compared with pines. It would be difficult to state the
comparative quickness of growth of the various timber trees, as so
much depends on soil, situation, and treatment; it also varies
considerably at different stages of their growth. It is well known,
that in proper soil, black Italian poplar, Salix alba, and red wood
willow, exceed all others.
As, for naval use, it is not the quickness of growth and bulk of the
timber altogether, but of the matured timber alone, which is of
consequence—we give a view of the number of growths or annual
circles of sap-wood (the useless part), which the main stems of
several kinds of trees presented. Most of those we examined had a
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com