Reinforcement Learning for Finance: Solve Problems in Finance with CNN and RNN Using the TensorFlow Library 1st Edition Samit Ahlawat - Download the full ebook version right now
Reinforcement Learning for Finance: Solve Problems in Finance with CNN and RNN Using the TensorFlow Library 1st Edition Samit Ahlawat - Download the full ebook version right now
https://ebookmass.com/product/python-for-finance-mastering-data-
driven-finance-2nd-edition/
https://ebookmass.com/product/foundations-of-finance-pearson-series-
in-finance/
https://ebookmass.com/product/machine-learning-for-risk-calculations-
a-practitioners-view-the-wiley-finance-series-1st-edition-ignacio-
ruiz/
https://ebookmass.com/product/978-1285429649-principles-of-finance-
finance-titles-in-the-brigham-family/
https://ebookmass.com/product/fix-it-see-and-solve-the-problems-of-
digital-healthcare-1st-edition-thimbleby/
Reinforcement
Learning for
Finance
Solve Problems in Finance with
CNN and RNN Using the
TensorFlow Library
—
Samit Ahlawat
Reinforcement
Learning for Finance
Solve Problems in Finance
with CNN and RNN Using
the TensorFlow Library
Samit Ahlawat
Reinforcement Learning for Finance: Solve Problems in Finance with CNN
and RNN Using the TensorFlow Library
Samit Ahlawat
Irvington, NJ, USA
Acknowledgments�������������������������������������������������������������������������������xi
Preface����������������������������������������������������������������������������������������������xiii
Introduction����������������������������������������������������������������������������������������xv
Chapter 1: Overview�����������������������������������������������������������������������������1
1.1 Methods for Training Neural Networks�����������������������������������������������������������2
1.2 Machine Learning in Finance��������������������������������������������������������������������������3
1.3 Structure of the Book��������������������������������������������������������������������������������������4
v
Table of Contents
vi
Table of Contents
vii
Table of Contents
Bibliography�������������������������������������������������������������������������������������403
Index�������������������������������������������������������������������������������������������������411
viii
About the Author
Samit Ahlawat is Senior Vice President in
Quantitative Research, Capital Modeling, at
JPMorgan Chase in New York, USA. In his
current role, he is responsible for building
trading strategies for asset management
and for building risk management models.
His research interests include artificial
intelligence, risk management, and
algorithmic trading strategies. He has given CQF Institute talks on artificial
intelligence, has authored several research papers in finance, and holds a
patent for facial recognition technology. In his spare time, he contributes
to open source code.
ix
Acknowledgments
I would like to express my heartfelt appreciation for my friends and
coworkers, in academia and the workplace, who encouraged me to write
this book.
xi
Visit https://ebookmass.com today to explore
a vast collection of ebooks across various
genres, available in popular formats like
PDF, EPUB, and MOBI, fully compatible with
all devices. Enjoy a seamless reading
experience and effortlessly download high-
quality materials in just a few simple steps.
Plus, don’t miss out on exciting offers that
let you access a wealth of knowledge at the
best prices!
Preface
When I began using artificial intelligence tools in quantitative financial
research, I could not find a comprehensive introductory text focusing on
financial applications. Neural network libraries like TensorFlow, PyTorch,
and Caffe had made tremendous contributions in the rapid development,
testing, and deployment of deep neural networks, but I found most
applications restricted to computer science, computer vision, and robotics.
Having to use reinforcement learning algorithms in finance served as
another reminder of the paucity of texts in this field. Furthermore, I found
myself referring to scholarly articles and papers for mathematical proofs of
new reinforcement learning algorithms. This led me to write this book to
provide a one-stop resource for Python programmers to learn the theory
behind reinforcement learning, augmented with practical examples drawn
from the field of finance.
In practical applications, reinforcement learning draws upon deep
neural networks. To facilitate exposition of topics in reinforcement
learning and for continuity, this book also provides an introduction to
TensorFlow and covers neural network topics like convolutional neural
networks (CNNs) and recurrent neural networks (RNNs).
Finally, this book also introduces readers to writing modular, reusable,
and extensible reinforcement learning code. Having worked on developing
trading strategies using reinforcement learning and publishing papers,
I felt existing reinforcement learning libraries like TF-Agents are tightly
coupled with the underlying implementation framework and do not
xiii
Preface
xiv
Introduction
Reinforcement learning is a rapidly growing area of artificial intelligence
that involves an agent learning from past experience of rewards gained
by taking specific actions in certain states. The agent seeks to learn a
policy prescribing the optimum action in each state with the objective of
maximizing expected discounted future rewards. It is an unsupervised
learning technique where the agent learns the optimum policy by past
interactions with the environment. Supervised learning, by contrast, seeks
to learn the pattern of output corresponding to each state in training
data. It attempts to train the model parameters in order to get a close
correspondence between predicted and actual output for a given set of
inputs. This book outlines the theory behind reinforcement learning
and illustrates it with examples of implementations using TensorFlow.
The examples demonstrate the theory and implementation details of the
algorithms, supplemented with a discussion of corresponding APIs from
TensorFlow and examples drawn from quantitative finance. It guides
a reader familiar with Python programming from basic to advanced
understanding of reinforcement learning algorithms, coupled with a
comprehensive discussion on how to use state-of-the-art software libraries
to implement advanced algorithms in reinforcement learning.
Most applications of reinforcement learning have focused on robotics
or computer science tasks. By focusing on examples drawn from finance,
this book illustrates a spectrum of financial applications that can benefit
from reinforcement learning.
xv
CHAPTER 1
Overview
Deep neural networks have transformed virtually every scientific human
endeavor – from image recognition, medical imaging, robotics, and self-
driving cars to space exploration. The extent of transformation heralded
by neural networks is unrivaled in contemporary human history, judging
by the range of new products that leverage neural networks. Smartphones,
smartwatches, and digital assistants – to name a few – demonstrate the
promise of neural networks and signal their emergence as a mainstream
technology. The rapid development of artificial intelligence and machine
learning algorithms has coincided with increasing computational power,
enabling them to run rapidly. Keeping pace with new developments in
this field, various open source libraries implementing neural networks
have blossomed. Python has emerged as the lingua franca of the artificial
intelligence programming community. This book aims to equip Python-
proficient programmers with a comprehensive knowledge on how to use
the TensorFlow library for coding deep neural networks and reinforcement
learning algorithms effectively. It achieves this by providing detailed
mathematical proofs of key theorems, supplemented by implementation of
those algorithms to solve real-life problems.
Finance has been an early adopter of artificial intelligence algorithms
with the application of neural networks in designing trading strategies
as early as the 1980s. For example, White (1988) applied a simple
neural network to find nonlinear patterns in IBM stock price. However,
recent cutting-edge research on reinforcement learning has focused
2
Chapter 1 Overview
3
Chapter 1 Overview
4
CHAPTER 2
Introduction to
TensorFlow
TensorFlow is an open source, high-performance machine learning library
developed by Google and released for public use in 2015. It has interfaces
for Python, C++, and Java programming languages. It has the option of
running on multiple CPUs or GPUs. TensorFlow offers two modes of
execution: eager mode that can be run immediately and graph mode
that creates a dependency graph and executes nodes in that graph only
where needed.
This book uses TensorFlow 2.9.1. Older TensorFlow constructs from
version 1 of the library such as Session and placeholder are not covered
here. Their use has been rendered obsolete in TensorFlow version 2.0 and
higher. Output shown in the code listings has been generated using the
PyCharm IDE’s interactive shell.
1 import tensorflow as tf
2
3 tensor = tf.constant([[list(range(3))],
4 [list(range(1, 4))],
5 [list(range(2, 5))]], dtype=tf.
float32)
6
7 print(tensor)
8
9 tf.Tensor(
10 [[[0. 1. 2.]]
11 [[1. 2. 3.]]
12 [[2. 3. 4.]]], shape=(3, 1, 3), dtype=float32)
6
Chapter 2 Introduction to TensorFlow
11 print(ar * ar)
12 Out[8]:
13 <tf.Tensor: id=2, shape=(2, 2), dtype=float32, numpy=
14 array([[1., 4.],
15 [4., 4.]], dtype=float32)>
16
17 # matrix multiplication C = tf.matmul(A, B) => cij =
sum_k (aik * bkj)
18 print(tf.matmul(ar, tf.transpose(ar)))
19
20 <tf.Tensor: id=5, shape=(2, 2), dtype=float32, numpy=
21 array([[5., 6.],
22 [6., 8.]], dtype=float32)>
23
24 # generic way of matrix multiplication
25 print(tf.einsum("ij,kj->ik", ar, ar))
26
27 <tf.Tensor: id=23, shape=(2, 2), dtype=float32, numpy=
28 array([[5., 6.],
29 [6., 8.]], dtype=float32)>
30
31 # cross product
32 print(tf.einsum("ij,kl->ijkl", ar, ar))
33
34 <tf.Tensor: id=32, shape=(2, 2, 2, 2),
dtype=float32, numpy=
35 array([[[[1., 2.],
36 [2., 2.]],
37 [[2., 4.],
38 [4., 4.]]],
39 [[[2., 4.],
7
Visit https://ebookmass.com today to explore
a vast collection of ebooks across various
genres, available in popular formats like
PDF, EPUB, and MOBI, fully compatible with
all devices. Enjoy a seamless reading
experience and effortlessly download high-
quality materials in just a few simple steps.
Plus, don’t miss out on exciting offers that
let you access a wealth of knowledge at the
best prices!
Other documents randomly have
different content
sur lui, roula lourdement sur ses gonds et donna ouverture à la
chambre destinée à milady.
D’un seul regard, la prisonnière embrassa l’appartement dans ses
moindres détails.
C’était une chambre dont l’ameublement était à la fois propre à
une prison et propre à une habitation d’homme libre; cependant des
barreaux aux fenêtres et des verrous extérieurs à la porte décidaient
le procès en faveur de la prison.
Un instant toute la force d’âme de cette créature l’abandonna;
elle tomba sur un fauteuil, croisant les bras, baissant la tête, et
s’attendant à chaque instant à voir entrer un juge pour l’interroger.
Mais personne n’entra, que deux ou trois soldats de marine qui
apportèrent les malles et les caisses, les déposèrent dans un coin et
se retirèrent sans rien dire.
L’officier présidait à tous les détails avec le même calme que
milady lui avait constamment vu, ne prononçant pas une parole lui-
même, et se faisant obéir d’un geste de sa main ou d’un coup de
son sifflet. On eût dit qu’entre cet homme et ses inférieurs la langue
parlée n’existait pas ou était devenue inutile.
Enfin milady n’y put tenir plus longtemps, elle rompit le silence.
—Au nom du ciel, monsieur! s’écria-t-elle, que veut dire tout ce
qui se passe? Fixez mes irrésolutions; j’ai du courage pour tout
danger que je prévois, pour tout malheur que je comprends. Où
suis-je et que suis-je ici? suis-je libre: pourquoi ces barreaux et ces
portes? suis-je prisonnière: quel crime ai-je commis?
—Vous êtes ici dans l’appartement qui vous est destiné, madame.
J’ai reçu l’ordre d’aller vous prendre en mer et de vous conduire en
ce château: cet ordre, je l’ai accompli, je crois, avec toute la rigidité
d’un soldat, mais aussi avec toute la courtoisie d’un gentilhomme. Là
se termine, du moins jusqu’à présent, la charge que j’avais à remplir
près de vous, le reste regarde une autre personne.
—Et cette autre personne, quelle est-elle? demanda milady; ne
pouvez-vous me dire son nom?...
En ce moment on entendit par les escaliers un grand bruit
d’éperons; quelques voix passèrent et s’éteignirent, et le bruit d’un
pas isolé se rapprocha de la porte.
—Cette personne, la voici, madame, dit l’officier en démasquant
le passage, et en se rangeant dans l’attitude du respect et de la
soumission.
En même temps, la porte s’ouvrit; un homme parut sur le seuil
de la porte.
Il était sans chapeau, portait l’épée au côté, et froissait un
mouchoir entre ses doigts.
Milady crut reconnaître cette ombre dans l’ombre; elle s’appuya
d’une main sur le bras de son fauteuil, et avança la tête comme pour
aller au-devant d’une certitude.
Alors l’étranger s’avança lentement; et, à mesure qu’il s’avançait
en entrant dans le cercle de lumière projeté par la lampe, milady se
reculait involontairement.
Puis, lorsqu’elle n’eut plus aucun doute:
—Eh quoi! mon frère! s’écria-t-elle au comble de la stupeur, c’est
vous?
—Oui, belle dame! répondit lord Winter en faisant un salut moitié
courtois, moitié ironique, moi-même.
—Mais alors, ce château?
—Est à moi.
—Cette chambre?
—C’est la vôtre.
—Je suis donc votre prisonnière?
—A peu près.
—Mais c’est un abus affreux de la force!
—Pas de grands mots; asseyons-nous, et causons tranquillement,
comme il convient de faire entre un frère et une sœur.
Puis, se retournant vers la porte, et voyant que le jeune officier
attendait ses derniers ordres:
—C’est bien, dit-il, je vous remercie; maintenant, laissez-nous,
monsieur Felton.
XX
ENTRE FRÈRE ET SŒUR
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookmass.com