Numerical Methods Using Kotlin: For Data Science, Analysis, and Engineering 1st Edition Haksun Li download
Numerical Methods Using Kotlin: For Data Science, Analysis, and Engineering 1st Edition Haksun Li download
https://ebookmass.com/product/numerical-methods-using-
kotlin-for-data-science-analysis-and-engineering-1st-
edition-haksun-li/
https://ebookmass.com/product/numerical-methods-using-kotlin-for-data-
science-analysis-and-engineering-1st-edition-haksun-li/
https://ebookmass.com/product/numerical-methods-in-environmental-data-
analysis-moses-eterigho-emetere/
https://ebookmass.com/product/spatial-analysis-using-big-data-methods-
and-urban-applications-yamagata/
https://ebookmass.com/product/using-r-for-data-analysis-in-social-
sciences-a-research-project-oriented-approach-li/
Statistical Methods for Survival Data Analysis 3rd Edition
Lee
https://ebookmass.com/product/statistical-methods-for-survival-data-
analysis-3rd-edition-lee/
https://ebookmass.com/product/exploratory-data-analysis-using-r-1st-
edition-ronald-k-pearson/
■
■Chapter 1: Introduction to Numerical Methods in Kotlin/S2������������������������������� 1
1.1. Library Design��������������������������������������������������������������������������������������������������������� 4
1.1.1. Class Parsimony������������������������������������������������������������������������������������������������������������������������� 4
1.1.2. Java vs C++ Performance���������������������������������������������������������������������������������������������������������� 5
1.3. S2��������������������������������������������������������������������������������������������������������������������������� 11
1.3.1. R Replacement�������������������������������������������������������������������������������������������������������������������������� 11
1.3.2. Python Replacement������������������������������������������������������������������������������������������������������������������ 23
1.3.3. S2 Online IDE����������������������������������������������������������������������������������������������������������������������������� 26
■
■Chapter 2: Linear Algebra����������������������������������������������������������������������������������� 35
2.1. Vector��������������������������������������������������������������������������������������������������������������������� 39
2.1.1. Element-Wise Operations���������������������������������������������������������������������������������������������������������� 40
2.1.2. Norm������������������������������������������������������������������������������������������������������������������������������������������ 44
2.1.3. Inner Product and Angle������������������������������������������������������������������������������������������������������������ 45
v
■ Table of Contents
2.2. Matrix��������������������������������������������������������������������������������������������������������������������� 47
2.2.1. Matrix Operations���������������������������������������������������������������������������������������������������������������������� 50
2.2.2. Element-Wise Operations���������������������������������������������������������������������������������������������������������� 51
2.2.3. Transpose���������������������������������������������������������������������������������������������������������������������������������� 53
2.2.4. Matrix Multiplication������������������������������������������������������������������������������������������������������������������ 54
2.2.5. Rank������������������������������������������������������������������������������������������������������������������������������������������ 62
2.2.6. Determinant������������������������������������������������������������������������������������������������������������������������������� 63
2.2.7. Inverse and Pseudo-Inverse������������������������������������������������������������������������������������������������������ 64
2.2.8. Kronecker Product��������������������������������������������������������������������������������������������������������������������� 68
vi
■ Table of Contents
■
■Chapter 3: Finding Roots of Equations�������������������������������������������������������������� 141
3.1. An Equation of One Variable���������������������������������������������������������������������������������� 141
3.2. Jenkins-Traub Algorithm��������������������������������������������������������������������������������������� 143
3.3. The Bisection Method�������������������������������������������������������������������������������������������� 144
3.4. Brent’s Method������������������������������������������������������������������������������������������������������ 148
3.4.1. Linear Interpolation Method, False Position Method, Secant Method�������������������������������������� 148
3.4.2. Inverse Quadratic Interpolation������������������������������������������������������������������������������������������������ 150
3.4.3. Brent’s Method Implementation����������������������������������������������������������������������������������������������� 151
■
■Chapter 4: Finding Roots of System of Equations��������������������������������������������� 159
4.1. System of Equations��������������������������������������������������������������������������������������������� 159
4.2. Finding Roots of Systems of Two Nonlinear Equations����������������������������������������� 162
4.3. Finding Roots of Systems of Three or More Equations����������������������������������������� 165
■
■Chapter 5: Curve Fitting and Interpolation�������������������������������������������������������� 169
5.1. Least Squares Curve Fitting���������������������������������������������������������������������������������� 170
5.2. Interpolation���������������������������������������������������������������������������������������������������������� 174
5.2.1. Linear Interpolation������������������������������������������������������������������������������������������������������������������ 174
5.2.2. Cubic Hermite Spline Interpolation������������������������������������������������������������������������������������������� 177
5.2.3. Cubic Spline Interpolation�������������������������������������������������������������������������������������������������������� 181
5.2.4. Newton Polynomial Interpolation���������������������������������������������������������������������������������������������� 184
■
■Chapter 6: Numerical Differentiation and Integration��������������������������������������� 197
6.1. Numerical Differentiation�������������������������������������������������������������������������������������� 198
6.2. Finite Difference���������������������������������������������������������������������������������������������������� 199
6.2.1. Forward Difference������������������������������������������������������������������������������������������������������������������� 200
6.2.2. Backward Difference���������������������������������������������������������������������������������������������������������������� 201
vii
■ Table of Contents
viii
■ Table of Contents
■
■Chapter 7: Ordinary Differential Equations�������������������������������������������������������� 267
7.1. Single-Step Method���������������������������������������������������������������������������������������������� 267
7.1.1. Euler’s Method (Polygon Method)��������������������������������������������������������������������������������������������� 269
7.1.2. Runge-Kutta Family������������������������������������������������������������������������������������������������������������������ 276
7.1.3. Convergence����������������������������������������������������������������������������������������������������������������������������� 282
7.1.4. Stability������������������������������������������������������������������������������������������������������������������������������������� 283
■
■Chapter 9: Unconstrained Optimization������������������������������������������������������������� 335
9.1. Brute-Force Search����������������������������������������������������������������������������������������������� 335
9.2. C2OptimProblem���������������������������������������������������������������������������������������������������� 339
9.3. Bracketing Methods���������������������������������������������������������������������������������������������� 340
9.3.1. Fibonacci Search Method��������������������������������������������������������������������������������������������������������� 342
9.3.2. Golden-Section Search������������������������������������������������������������������������������������������������������������� 344
9.3.3. Brent’s Search�������������������������������������������������������������������������������������������������������������������������� 345
ix
■ Table of Contents
■
■Chapter 10: Constrained Optimization��������������������������������������������������������������� 373
10.1. The Optimization Problem����������������������������������������������������������������������������������� 373
10.1.1. General Optimization Algorithm���������������������������������������������������������������������������������������������� 375
10.1.2. Constraints����������������������������������������������������������������������������������������������������������������������������� 376
x
■ Table of Contents
■
■Chapter 11: Heuristics��������������������������������������������������������������������������������������� 473
11.1. Penalty Function Method������������������������������������������������������������������������������������� 475
11.2. Genetic Algorithm������������������������������������������������������������������������������������������������ 478
11.2.1. Encoding��������������������������������������������������������������������������������������������������������������������������������� 479
11.2.2. Fitness Function��������������������������������������������������������������������������������������������������������������������� 479
11.2.3. Initial Population��������������������������������������������������������������������������������������������������������������������� 480
11.2.4. The Operation Process������������������������������������������������������������������������������������������������������������ 480
11.2.5. Selection��������������������������������������������������������������������������������������������������������������������������������� 481
11.2.6. Crossover�������������������������������������������������������������������������������������������������������������������������������� 481
11.2.7. Mutation��������������������������������������������������������������������������������������������������������������������������������� 482
11.2.8. Differential Evolution�������������������������������������������������������������������������������������������������������������� 487
xi
■ Table of Contents
xii
■ Table of Contents
■
■Chapter 13: Random Numbers and Simulation������������������������������������������������� 647
13.1. Uniform Random Number Generators����������������������������������������������������������������� 648
13.1.1. Linear Congruential Methods������������������������������������������������������������������������������������������������� 648
13.1.2. Mersenne Twister������������������������������������������������������������������������������������������������������������������� 651
■
■Chapter 14: Linear Regression�������������������������������������������������������������������������� 689
14.1. Ordinary Least Squares��������������������������������������������������������������������������������������� 692
14.1.1. Assumptions��������������������������������������������������������������������������������������������������������������������������� 693
14.1.2. Model Properties�������������������������������������������������������������������������������������������������������������������� 694
14.1.3. Residual Analysis�������������������������������������������������������������������������������������������������������������������� 695
14.1.4. Influential Point����������������������������������������������������������������������������������������������������������������������� 699
14.1.5. Information Criteria���������������������������������������������������������������������������������������������������������������� 704
14.1.6. NM Dev Linear Regression Package��������������������������������������������������������������������������������������� 705
xiii
■ Table of Contents
■
■Chapter 15: Time-Series Analysis��������������������������������������������������������������������� 737
15.1. Univariate Time Series���������������������������������������������������������������������������������������� 743
15.1.1. Stationarity����������������������������������������������������������������������������������������������������������������������������� 747
15.1.2. Autocovariance����������������������������������������������������������������������������������������������������������������������� 747
15.1.3. Autocorrelation����������������������������������������������������������������������������������������������������������������������� 749
15.1.4. Partial Autocorrelation������������������������������������������������������������������������������������������������������������ 750
15.1.5. White Noise Process and Random Walk��������������������������������������������������������������������������������� 752
15.1.6. Ljung-Box Test for White Noise����������������������������������������������������������������������������������������������� 753
15.1.7. Model Decomposition������������������������������������������������������������������������������������������������������������� 755
■
■Index ������������������������������������������������������������������������������������������������������������������ 883
xiv
About the Author
xv
About the Technical Reviewer
Mark Wickham is a software developer based in Dallas. Prior to returning to Texas in 2020, Mark lived in
Beijing for nearly 20 years, where he worked for Motorola, Inc., as a software engineering manager, and in
various program management and product management roles for the Asia-Pacific region. Mark got his start
in Java and Android when Motorola produced the first Java-based mobile phone and he wanted to write his
own apps.
Mark was a regular speaker at some of the well-known Android developer conferences and has written
two books for Apress, Practical Java Machine Learning and Practical Android.
Before embarking on his professional career, Mark obtained a degree in physics from Creighton
University in Omaha, with a major in computer science and a minor in mathematics. Mark later obtained
an MBA from the University of Washington and also studied at the Hong Kong University of Science and
Technology.
An Iowa native, in his free time, Mark enjoys gravel bike riding, sports photography, and his latest
new passion, disc golf. For more information about Mark, connect with him at linkedin.com/in/
mark-j-wickham.
xvii
Acknowledgments
I never knew that writing a book would be such a tremendous effort, especially for this book with such wide
coverage. It started out as a one-year project and then was extended to become an almost two-year project.
For eight months in 2021, I basically did nothing other than write this book, pretty much every day. What
would be a better thing to do during the indefinite COVID-19 lockdown?
This book covers a lot of topics in basic college applied mathematics, supplemented with a lot of sample
code and more than 300 images for illustration. It was a tremendous effort to put the materials together.
I want to thank my team for their incredible support throughout the writing of this manuscript. They are
amazing and this book would not have happened without them: Kehao Feng, Jingqi Guo, Shihao Liu, Yang
Lu, Hengyu Luo, and Dr. Xueying Ni.
Although we have never met in person, I thank Steve Anglin for inviting me to write this book,
something that I have wanted to do for a long time but never found sufficient motivation to do. I thank you
Mark Wickham for carefully reviewing my book and making a lot of good comments and suggestions. Last
but not least, I must thank Mark Powers for pushing me to submit the manuscript. He provided a huge push
for me to sit down at my computer and start typing and meet the always-missed-and-always-late deadlines.
I thank you all for your patience!
xix
Preface
When I was a boy, I liked to organize my math notes. I did not really like math (or any studies) at all until
I met my first math teacher, who enlightened me. It was Professor Bīn Hé (何斌) who showed me how
interesting math could be and taught me math at the Sun Yat-sen University in the summers when I was in
junior high school. To him, I am forever grateful. Mathematical training shaped the way I think and opened
the door to learning other science subjects like physics. Those summers changed my life from being a
mediocre student to a top one.
I found mathematics interesting because there are structures in numbers. For example, prime numbers
build all numbers; there are no “holes” in the real number line. There are so many ways to prove the same
thing, e.g., the Pythagorean theorem. There are again structures and relationships among geometric shapes,
as well as between shapes and numbers. It is fascinating why. I thought I was pretty good at mathematics
until my first year taking an honor’s class in real analysis at the university. Some of my classmates were way
beyond my league. I did not see myself becoming a mathematician, so I switched to computer science in
graduate school, thinking that it might be easier.
Studying computer science is never about learning to program. It turned out that I was a pretty good
coder, or so I thought, after coding C++ almost a decade at universities. I had the good aptitude of spending
hours and even days doing nothing other than debugging a segmentation fault, fixing a memory leak, or
comparing numbers down to the last decimal place. (I still remember when it took me three days to figure
out a very obscure bug at UBS.) It turned out that I was ignorantly over-confident when my first mentor
introduced design patterns to me. Dr. Anil Kumar asked me to design a singleton during a job interview and
guided me through designing the double-checked locking pattern. Luckily, he still hired me onto his team
for my first job. I am thankful for his training, guidance, and mentorship, which jump-started my career of
becoming a professional quantitative trader.
My training in mathematics and computer science, with a specialization in artificial intelligence, has
been powering my career as a quantitative trader and applying mathematics to solve business problems. It is
a rare combination of skills. Most mathematicians cannot code (Excel/MATLIB/R is not really professional
coding), and most programmers do not read mathematics. We often needed to translate R code into Java
code and back and forth between the mathematicians who created the trading models and the programmers
who implemented them in an order-execution system. So, one day I decided to make myself useful by
creating a programming library of mathematics, using both of my skills. This is kind of a continuation of my
childhood hobby of organizing my math notes, because now I am organizing math algorithms in a more
useful form. The objective is that quantitative programmers can code mathematics in the same language
that they use to build applications. There will be no more “lost in translation” issues.
I was lucky to hire Dr. Ken Yiu in 2007, starting out as an intern and then going full-time, who is a
cofounder of the startup. Ken is the best software engineer I have ever met. He is smart, quick-witted,
careful, and above all responsible. I cannot thank him enough for traveling with me through this adventure
of learning and building the math library, trading strategies, and systems. He has taught me a lot about
computer science and programming.
In 2008, Dr. Kevin Sun joined the team as an intern. I was fortunate to have Kevin later join the startup as
another cofounder. He is now the brains behind our team and is the key person creating many mathematical
models in trading and gambling. He has excellent mathematical-thinking and problem-solving skills. I am still
xxi
■ Preface
impressed by how he could back then (and still can) solve in a nice and elegant way just about every problem
we threw at him. He always saves the team before deadlines with his incredible math and R skills. He has
taught me a lot about mathematics and statistics.
Also, I am forever grateful to my brother, Kevin, for all his moral support, teamwork, contributions,
encouragement, advice, connections, precious time, and financial assistance through all the ups and
downs, laughs and tears, and hopes on this journey with me. Nothing would have been possible without his
tremendous effort and resources. I thank God for having him in life.
This is the story of how NM Dev, the math library, started. Today, the library has been adopted by
big-name commercial organizations as well as academic institutions. NM Dev is the cornerstone of all
the work that we do. We build trading strategies, financial analytics (AlgoQuant), fixed-income analytics
(SuperCurve), data-driven decision-making systems (NM Optim), and high-performance solvers and
optimizers (NM Solver) based on this Java numerical programming library.
Source Code
All source code used in this book can be downloaded from github.com/apress/numerical-methods-kotlin.
xxii
CHAPTER 1
When I coded the first line of a matrix multiplication algorithm, the classical O(n3) version, in the attic of my
London apartment during an evening back in 2009, it was nothing more than a hobby. I was coding a trading
strategy in Java but could not find a good Java math library to use (namely, one that was professionally
maintained, well documented with examples, solidly object-oriented, and had an extensive open data
structure). A large part of the strategy code was developed in R/MATLAB and the trade/order execution
system was in Java. However, making Java talk to R/MATLAB and vice versa was a nightmare, let alone the
many other IT nuisances (e.g., multi-thread safety, multi-instances, debugging, R/MATLAB was very slow,
and the mysterious who-knows-why-not-working issues).
I thought that the world would benefit from having a Java math library much like the then-ubiquitous
“numerical recipes in C”. One easy way to create one was to (auto) translate the C code into Java. Yet, this
gave me a bunch of functions (procedures) that did only what the functions did. There was no data structure
or module or (sub-)algorithm that I could reuse to build my new data structures or algorithms. A vector
should not just be a bunch of numbers or an array. A vector is a mathematical concept that is well-defined
with properties and operations. Any algorithm that works with vectors should understand them, and not
have to re-code the properties and operations.
While there were a few Java numerical libraries at the time, most notably Apache Commons Math,
none of them was designed to be high performance. Modern computing has come a long way since
the FORTRAN/C era. We now have concurrent programming, more efficient algorithms (for matrix
multiplication, etc.), new hardware like GPUs, and so on. Simple translation of FORTRAN/C code or
textbook implementation of algorithms would not leverage these advances. Performance would be
very slow. We needed a multi-threading, highly optimized data structure as well as efficient algorithms.
For example, instead of using the simple O(n3) algorithm for matrix multiplication, we implemented a
parallelized version of the Strassen algorithm. Due to a lot of team effort twisting, experimenting, and
testing things during these ten years, the end result is that we now have arguably the fastest linear algebra
implementation in Java.
Another problem that I had with the existing Java math libraries was that they were very limited in
scope. A quantitative trading strategy can be as simple as a moving average crossover, but it can also be
very complicated, involving solving stochastic differential equations. It certainly involves a lot of statistical
analysis. Last but not least, it also takes a suite of solvers to optimize things like strategy parameters.
Pretty much all statistical procedures involve some optimization, for example, regression and maximum
likelihood. There is no free or cheap optimizer that is good and is in Java. Professional ones such as Gurobi
and Mosek are expensive.
Something might have hit my head and made me do the unthinkable. I resigned from my trader job
with the investment bank and embarked on a great adventure. I set out to create a Java math library that
was well designed with data structure and software engineering, easy-to-read for humans (not machines),
leveraged modern computing technologies, enjoyed high performance, and was extensive, including linear
algebra, Calculus, statistics, time series analysis, optimization, stochastic differential equations, and so on.
I didn’t realize how difficult this would be when I quit my job. I didn’t know how many Herculean tasks
there were to overcome. I just thought that it would be fun to create something to (hopefully) share with
the world. When I was a teen, I used to write my own math books by organizing my mathematics notes. I
supposed then that organizing algorithms in code would be much more useful, a sort of continuation of my
childhood hobby.
It turned out that building a math library, especially a good one, was very challenging. It was much
more than translating math formulas into code. First, we needed to understand every detail in the
algorithms. For many of them, we read the original publications. Some old publications were written in an
ancient and arcane language that was very difficult to understand. Many papers had details missing. Many
papers had errors and typos. For each algorithm that we implemented, we had to re-derive every step in the
mathematics to make sure that we understood all the details and that the algorithm was correct. This was
nothing like learning from textbooks, which are usually well written and illustrated with plenty of examples.
There are many concepts that we are used to using without even thinking about the details. For
example, we use eigenvalues and eigenvectors of a matrix all the time, but few of us care how decomposing
a matrix actually works. To code a linear algebra package, we needed to understand the many different
decompositions and factorizations of a matrix, their pros and cons, precision, numerical stability, debugging
and testing, API design, and user friendliness — all little details. This process was non-trivial.
Second, we had to verify that our code was correct. If the paper came with examples, we reproduced
them to make sure that our results matched theirs. They were usually very few simple illustrative examples.
We needed to try for a wide range of test cases, corner cases, stress tests, and performance tests. Our
benchmark was R. We would always match our results with Rs. When there were differences, we had to
explain every one of them. We were very serious about the accuracy of our code and were constantly
fixing bugs in each release. In some cases, our code was even more accurate than the Rs. For example,
we didn’t know that R has for Johansen tests only a few critical points — e.g., 10%, 5%, 1% — and did a
linear interpolation between them for the other values (from the R’s source code). On the other hand, we
computed the asymptotic distribution for Johansen tests using the exact distribution functions. That was
many hours of debugging and reading the R source code.
Third, we designed the software so that it was user friendly and in line with good software engineering
principles. Most popular math libraries, such as Netlib, were written decades ago in FORTRAN/C. They were
neither user friendly nor used good programming (because there was not much computer science at that
time). One most annoying thing to me was that a function often took dozens of parameters with impossible-
to-decipher-and-remember names (because the concept of “object” and data structure did not exist back
then; names were limited to eight characters). I often could not remember what the variables meant, missed
putting in some arguments, and messed up their orders. They were not user friendly by modern standards.
The code was often very difficult to read. Good code should read like English and should be easy to
maintain by other professional programmers without the author sitting next to them. When we coded our
math library, we made every effort to have the lines mirror the paper so that another person can follow our
logic by comparison. This was so that others could fix our bugs, extend our work, translate it to another
language (e.g., C#), and port it to other hardware (e.g., GPU). They needed to understand our code to do so.
My opinion was that it was next-to-impossible to do this with the decades’ old FORTRAN/C code. I bet that
once those authors were gone, no one could do anything with that code other than run it, having faith that it
had no bugs. I coded up ALGORITHM AS 288 in Java from reading the original paper. I tried to understand
the AS 288 FORTRAN code (see Figure 1-1) for comparison, but have still failed to do so to this date. The
FORTRAN code just seems to move variables around for no obvious reason and somehow gets it right
magically.
2
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
Figure 1-1. It is very hard to remember the meanings and order of A, B, C, K, N, X, Y (AS108)
Our first attempt in building such a library was SuanShu, named after one of the earliest known Chinese
mathematical treatises, 算数书, which was written around 200 B.C. Since 2009, it was a decade of effort. It
started as a basic linear algebra package and grew to be an extensive library covering Calculus, optimization
algorithms, statistical analysis, differential equations, and so on. It started being my personal hobby and
then dozens of experts, professors, PhDs, students, and practitioners around the world contributed to the
library. It was adopted by some big-name commercial organizations as well as academic institutions. In
2018, we decided to discontinue the sale and support for SuanShu. We made the codebase open-source as of
June 2012.
Leveraging our decade of experience with numerical programming, NM Dev is a fresh new numerical
library that succeeds SuanShu. We have collected a large amount of feedback from users and bug reports in
the last ten years. We want to have a new software foundation and architecture to account for all these good
suggestions.
3
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
1.1. Library Design
The objective of SuanShu and hence NM Dev is to enable very easy programming of engineering
applications. Programmers can program mathematics in such a way that the source code is solidly object-
oriented and individually testable. NM Dev source code adheres to the strictest coding standards so that it is
readable, maintainable, and can be easily modified and extended.
NM Dev revolutionizes how numerical computing is traditionally done, for example, Netlib and gsl.
The repositories of these popular and “standard” libraries are collections of ad hoc source code in old
languages, such as FORTRAN and C. One problem with such code is that it is not readable (for most modern
programmers), hence it’s unmaintainable. For example, it is quite a challenge to understand AS 288,
let alone improve on it. Other problems include, but are not limited to, the lack of data structure, duplicated
code, being entirely procedural, very bad variable naming, abuse of GOTO, the lack of test cases, insufficient
documentation, the lack of IDE support, inconvenient linking to modern languages such as Java, unfriendly
to parallel computing, and so on.
To address these problems, NM Dev designs a framework of reusable math components (not
procedures) so that programmers can put components together like Legos to build more complex
algorithms. NM Dev is written anew so that it conforms to the modern programming paradigms, such as
variable naming, code structuring, reusability, readability, maintainability, as well as software engineering
procedures. To ensure very high quality of the code and very few bugs, NM Dev has thousands of unit test
cases that run daily.
1.1.1. Class Parsimony
We decided to make the class library as parsimonious as possible to avoid method pollution. This is inspired
by the jMatrices’ whitepaper.1 The challenge is to organize the methods by minimal and correct packages.
We illustrate this with the NM Dev matrix package.
The Matrix class has only 26 methods, nine of which are constructors and the related methods, three
are overrides for the AbstractMatrix interfaces, and eight are overrides for the MatrixSpace interfaces.
Only six of them are class-specific, to make calling these methods convenient for the user. The other dozens
of matrix operations — such as the different factorizations, properties like rank, and transformations like
inverse — are grouped into multiple classes and packages. In most cases, each of these operations is a class
on its own. For instance, the inverse operation is a class inheriting from Matrix. The constructor takes as
input the matrix to invert. For example, to find the inverse for the following:
1 2 3
A 6 5 4
8 7 9
val A = DenseMatrix(arrayOf(
doubleArrayOf(1.0, 2.0, 3.0),
doubleArrayOf(6.0, 5.0, 4.0),
doubleArrayOf(8.0, 7.0, 9.0)))
val Ainv = Inverse(A)
Ainv
1
http://jmatrices.sourceforge.net/
4
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
3x3
[,1] [,2] [,3]
[1,] -0.809524, -0.142857, 0.333333,
[2,] 1.047619, 0.714286, -0.666667,
[3,] -0.095238, -0.428571, 0.333333,
It is important to note that the Ainv object is of the Matrix class; it’s created as a new instance of the
Inverse class. In summary, we choose to have hundreds of classes, rather than to have one class with
hundreds of methods. Each class is kept deliberately short. This class parsimony principle is a key design
decision that guides the whole library development.
1.2. Java Scripting
To do data science and analysis, the main problem with using Java is that you cannot work on the data in
interactive mode. That is, I would like to type a command to work on the data, such as to extract the first six
rows from a table, take a look at the result, think about what to do next, and then type another command.
The ability to work in interactive mode makes MATLAB/R the most popular tool in data analysis. See
Figure 1-2.
5
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
Figure 1-2. Print out the first six rows in a data set in R
By contrast, when using Java, you always have to write, compile, and then run the whole program. There are
many problems with this approach. First, it is very annoying to always have to repeatedly compile, deploy, and
execute the code. Second, Java is verbose. It has too much boilerplate code. There are sections of unnecessary
code that have to be included in many places with little or no alteration. You need to write a lot of code even
for simple things like printing out a number. Third, it is very time consuming to run the whole program from
the beginning each time. Fourth, you often do not know what program you want to write without examining,
understanding, and experimenting with the data. Java does not allow programmers to try different things with
the data, such as look the first six rows versus the first hundred rows. Fifth, you cannot pause a Java program in
the middle of a run to examine the intermediate results. In a nutshell, Java is not the right tool for prototyping
and developing data science code. You need a Java-based scripting language that lets you interact with data.
There are a few Java-based scripting languages that fully interoperate with existing Java code and libraries,
most notably Clojure, Scala, Groovy, and Kotlin. (I do not count JRuby and Jython because they are simply Java
implementation of Ruby and Python, respectively. Their goals are to make Ruby and Python code run on JVM.)
I do not like Clojure, a dynamic and functional dialect of the LISP programming language on the Java platform,
because there are too many parentheses to read and they hurt my eyes and brain.2 I can read English-like
2
My first class in computer science at the University of Chicago used the text book Structure and
Interpretation of Computer Programs by Harold Abelson, Gerald Jay Sussman, and Julie Sussman.
It was very annoying to have to parse all those parentheses in my head. It was also useless for my
career, as I have never once used LISP in my life after finishing that class. I would think that teaching
Java or C++ would be more useful and practical. I used C++ for my PhD and Java for my work.
6
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
sentences much faster than LISPs. Scala is a strong, statically typed general-purpose programming language
that supports both object-oriented programming and functional programming. Designed to be concise, many
of Scala’s design decisions are aimed to address the criticisms of Java. My problem with Scala is its learning
curve. I cannot figure out what the lines mean at the first glance, especially with all the symbols like :, ::, :::. I
want a language that I can pick up and use right away without spending time learning it.
Groovy is easy to learn and read. I did not spend any time learning it. I simply copied, pasted, and
modified the code and was able to get things working. Groovy is a Java-syntax-compatible object-oriented
programming language for the Java platform. It is both a static and dynamic language with features similar
to Python. It can be used as a programming language and as a scripting language, is compiled to Java Virtual
Machine (JVM) bytecode, and interoperates seamlessly with other Java code and libraries. Groovy uses a
curly-bracket syntax similar to Java’s. SuanShu used to support Groovy. For example, the following two-line
Groovy script evaluates a definite integral. See Figure 1-3. More examples can be found in Figure 1-4.
7
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
Unfortunately, the company Pivotal Software ceased sponsoring Groovy in April 2015. I was looking for
an alternative that was motivated by commercial interest with better resources and support.
8
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
In addition to its prominent use on Android, Kotlin is gaining traction in server-side development.
The Spring Framework officially added Kotlin support with version 5 on January 4, 2017. To further support
Kotlin, Spring has translated all its documentation to Kotlin and added built-in support for many Kotlin-
specific features, such as coroutines. In addition to Spring, JetBrains has produced a Kotlin-first framework
called Ktor for building web applications.
With all the commercial interests and community momentum, Kotlin is likely to remain a mainstream
language and be well supported. Kotlin will likely (and hopefully) stay for a very long time. Therefore, I chose
Kotlin as the scripting language to do math computing and data analysis.
One major advantage of Kotlin over Java is that Kotlin code is concise. It has little boilerplate code, is
shorter and more readable. Also, Kotlin is very easy to learn. I have never read any tutorials on the Internet
or books about Kotlin. I simply copy, paste, and modify existing snippets. This approach seems to work for
me and proves how easy Kotlin is to use and read. In fact, all sample code in this book was coded in this
fashion.
For example, to add two numbers, you do the following:
1 + 1
2 * 2
val A = 1.0
To print out a variable, you simply type its name. For example:
1.0
This statement says that x is a variable whose value can change. It is of type Double.
9
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
X = x + 1
x
2.0
for (i in 1..3) {
println(i)
}
1
2
3
1.0
2.0
3.0
4.0
5.0
add1(2.0)
10
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
3.0
There are many good books and tutorials on the Internet to learn how to code Kotlin. The official Kotlin
Foundation website has a lot of information about Kotlin, its development, news, and community.
https://kotlinlang.org
They also have an online course to get beginners started on coding Kotlin.
https://kotlinlang.org/docs/getting-started.html
1.3. S2
S2 is an online integrated development environment of data and analytics for data science and numerical
computing. It is developed by the company NM DEV PTE LTD. The official S2 website is:
http://nm.dev/s2
S2 contains all numerical algorithms in the NM Dev library and some mathematics, science, machine
learning, graphing, plotting, financial, and other commercial libraries. S2 lets users create their math
algorithms by calling all these built-in libraries in a simple manner using Kotlin.
1.3.1. R Replacement
S2 is designed to be a modern-day replacement for R. You can do a lot of things in S2 that you can do in R
and S2 does them faster. Moreover, R code runs inside only the R environment. It is very difficult to deploy
the code anywhere else such as in embedded devices like microwaves, automobiles, or space rockets. S2
code runs in any JVM environment. There are now 15 billion devices that run JVM!
S2 is an online IDE for numerical computing and coding numerical algorithms. It is like an online
calculator. Let’s start with 1+1, as shown in Figure 1-5.
With 1+1 working, you can do pretty much anything in numerical programming, which is just a more
complicated series of many 1+1 computations. For example, integration is shown in Figure 1-6.
11
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
S2 supports a few dozen type of graphs, charts, and plots. See Figures 1-7 through 1-12 for examples.
12
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
13
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
14
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
S2 has a very comprehensive statistics package. It supports almost all types of linear regressions and
their statistics, OLS, GLM, and logistics. See Figures 1-13 through 1-16 for examples.
15
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
16
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
17
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
Solvers are the foundation of the future of mathematics. They are the core of AI and Big Data analysis.
You need solvers for any problems that do not have a closed form solution. That is pretty much any modern
problem nowadays. S2 supports a full suite of all known standard optimization algorithms. See Figures 1-17
through 1-19 for examples.
18
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
19
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
S2 supports a few machine learning libraries, such as WEKA. It is simple to create and train a Neural
Network (NN) in S2. The example in Figure 1-20 is a simple script that trains an NN to learn the Black-
Scholes formula from a data set of stock prices and option prices.
20
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
21
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
The neural network converges in a few hundred epochs. The output is shown in Figure 1-21.
22
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
1.3.2. Python Replacement
S2 using Kotlin can challenge what Python can do in data science. The problems with Python are that
it’s scripting or interpreted language is slow, and it is very difficult to deploy code to other devices due to
numerous versioning of dependencies and an assorted array of libraries in FORTRAN, C, C++, and so on. S2
is fast and runs on the billions of devices with a JVM by copy-and-pasting jars. Python does two good things:
It treats scripting as a glue to combine many components to do data analysis, and it supports array/tensor
programming. S2 does those as well, but better.
Python is slow, very slow.3 Secondly, Python scripts run only in the Python environment. You cannot
port it to your phone, watch, router, automobile, rockets. Worst of all, Python deployment can be a
nightmare. It runs fine on your machine, but it can take a lot of effort to make it run on another person’s
machine due to the numerous versioning, compatibility between libraries, and machine dependent
executables. In contrast, S2 scripting, compiled to Java bytecode, is orders of magnitude faster than Python.
It runs on any (embedded) device that runs JVM, hence there is no deployment problem.
S2 scripting acts as the glue that combines many components, but with much better performance.
The following case shows a scheduling system built for a large steel manufacturing plant. The steps are as
follows:
1. Read the job data.
2. Read the machine data.
3. Schedule the jobs to the machines.
4. Plot the job-shop schedules to maximize utilization.
All these steps are done in an S2 script in 12 lines! See Figure 1-5. This same code can be deployed on
S2, on a stand-alone application, or on a cloud using REST.
3
See https://belitsoft.com/java-development-services/java-vs-python-tried-and-
true-vs-modern-and-new
23
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
24
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
The output is a job schedule and a Gantt chat, as shown in Figure 1-23.
The power of Python comes from these three libraries: scipy, numpy, and pandas. Together, they allow
users to put data in a high-dimensional array (aka tensor) so that they can dice, slice, cut, and sample the
tensor in however way they want. More importantly, they magically “convert” any Python script into parallel
execution code for high performance. They split a pandas DataFrame into several chunks, spawn a thread to
operate on each chunk, and puts them back together.
S2 supports exactly this kind of array/tensor programming for parallelization, and all kinds of
dissecting, slicing, cutting, dicing, sampling, massaging data frame using ND4J. We have a paralleled version
of the Black-Scholes formula application example. The formula is applied concurrently to all the rows
in the stock share price/option price table/array, in the same fashion that Python does with pandas (see
Figure 1-24). The code can be found at:
https://s21.nm.dev/hub/user-redirect/lab/tree/s2-public/Examples/Neutral%20Network/
Black_Scholes_learning_asynchronous.ipynb
25
Chapter 1 ■ Introduction to Numerical Methods in Kotlin/S2
Figure 1-24. S2 script to apply the Black-Scholes formula to a big table of stock share prices to compute the
option prices
https://s21.nm.dev/
S2 will start loading and create a workspace for your session (see Figure 1-25).
26
Other documents randomly have
different content
E trovar non potrei verso nè via
Che mi dessero certa e piena fede;
Massime in questo canto, ove la pia
Mente del sommo Dio sì ben provvede
Al mal di quella sfortunata e ria
Isola, fatta di Folletti sede:
Chè non può venir lor neppur in testa
Il frate co' giganti e la tempesta.
8
E qui bisognerebbe ch'io dicessi
Ogni minuzia fino ad un puntino.
Ma so che brevitade io vi promessi;
E più tosto restar senza un quattrino
Vo', che mancare a quello ch'io v'espressi.
Dirovvi dunque in mio schietto latino,
Che con le mogli lor Ricciardo e Guido
Sceser senza saperlo in su quel lido:
10
11
Acciocchè non istiate con pensiero,
E a lungo andare non m'esca di mente.
Riconosciuta adunque il campion nero
La sua bella Dorina ed innocente,
Più ratto assai che a lepre il can levriero,
Le corse a' piedi, e le chiese piangente
Perdon di quanto aveva e detto e fatto,
Reso per gelosia crudele e matto.
12
13
14
Quivi studiaro come disperati,
E si fecero bravi latinanti,
Nè fûro dal maestro mai frustati;
E andaron tanto con lo studio avanti,
Che dal vicino vescovo chiamati
Fûro, e promossi agli ordini più santi:
E da Tolon venivano a Marsiglia
Le genti per veder tal maraviglia.
15
16
17
Ma lasciamo per ora i missionari,
E parliamo del conte e di Rinaldo,
Che mentre erran per l'isola, e di vari
Casi van ragionando, da gran caldo
Presi son sì, che fan sospiri amari:
Nè il buon conte potendo star più saldo,
Dice a Rinaldo: Mi par questo loco,
S'io non m'inganno, l'Isola del foco.
18
19
20
Se questo fosse, cugin mio, l'inferno
(Disse Rinaldo), ci sarìa più folla:
E qui, fuor di noi due, niun altro scerno.
Allor, qual tin che per vinaccia bolla,
E di fuor gorgogliando e per l'interno,
Alza all'intorno or una or altra bolla;
Si senton sotto i piè la terra alzare,
E susurrar d'intorno e cigolare.
21
22
23
E di fiori e d'erbette si riveste
La terra da per tutto; e frutti e foglie
Mostran le piante in quelle parti e in queste:
Ed ogni augel la lingua al canto scioglie,
Da volgere in piacere le più meste,
E le più crude e tormentose doglie:
Ma quel che rallegrar li fece affatto,
Fu la comparsa di più ninfe a un tratto.
24
25
26
Deh, prima che ti colga il dì fatale,
E poca polve il cener tuo ricopra,
Lascia quest'arme, che a sì poco vale,
Ch'ogni nome perisce, ogni bell'opra,
E godi nosco. Anche il piacere ha l'ale;
Ma per goder, fatica non si adopra.
Però, se saggio sei, come tu mostri,
Spògliati, e vieni negli alberghi nostri.
27
28
29
E vuol gridare; ma cresce la piena,
Ed a Rinaldo pur passato ha il mento.
Onde pensate voi, donne, la pena
De' paladini, e l'atroce tormento
D'aver sì brutto pranzo e brutta cena.
Orlando pieno di crudel talento
Vuole ammazzarsi, ma non può morire;
Nè sa l'altro che farsi, o che si dire.
30
31
32
Rinaldo dà di mano alla forchetta.
Ed infila un fagiano, e quel sen vola;
Chiappa una starna, e mentre con gran fretta
La vuol tagliar per cacciarsela in gola,
Fugge, e con essa ogni altra pur sgambetta;
Talchè rimasta è la tovaglia sola.
Dice Orlando: Tu hai fatto molto presto!
Tace Rinaldo, e sta turbato e mesto.
33
34
35
Ma prima che comincin lo scongiuro,
Climene e Ricciardetto con Despina
Ecco, e Guidone il giovine sicuro,
Con l'altra gente che il bosco cammina:
E visto il frate in abito sì puro
Con que' due cherchi dalla cappellina,
Dieder 'n un riso sì spropositato,
Che Ferraù ne fu scandalizzato:
36
37
38
Ferraù grida: Da parte di Dio
Io vi comando, spiriti dannati,
Che danno non facciate al clero mio,
E stiate sotto me subordinati.
Ma quelli che di pugna hanno desìo,
Van lor sopra, e dan lor colpi spietati.
Ferrautte a quel dir dice ai giganti:
Meniam le mani, e non facciam più i santi;
39
40
41
E si fa segni di croce a bizzeffe;
Ma vedendo che punto non si muove,
Dice tra sè: Queste non son già beffe
Di spirti, che non reggono a tai prove:
E volle fare come il buon Gioseffe,
Fuggire; ma nel mentre che si move,
Climene piglia in mano il suo cordone,
Ed al romito vien la tentazione;
42
43
44
E don Fracassa anch'ei sèguita a dire
Parole sacre, tratte dal breviario;
Cioè che pensi come ha da morire;
E che non può pigliarsi un tale svario
Chi voto feo di castità soffrire.
Talchè principia sul suo calendario
Ferrautte ad averli tutti due;
E segni fa, che non ne può già piùe;
45
46
47
Ci vuol pur poco a mettere a romore
Il vicinato, e biasimare altrui,
E un frate lacerar vinto d'amore.
Figliuoli miei, che vi credete vui
Che il tonachino ci pari l'ardore
Che mandan fuori largamente dui
Occhi leggiadri, nè possano i frati
Diventare in niun tempo innamorati?
48
49
50
Così fatto avess'io quel dì fatale
Ch'io vinsi gli altri, e me vinse costei.
Ma chi potea pensar che tanto male
Da sì bel volto ritratto ne avrei?
Il pianger dopo il fatto a nulla vale;
Nè il mio danno fuggir seppi o potei.
Sola mercè del guasto mio consiglio;
Chè veggo il bene, ed al peggior m'appiglio.
51
52
53
E quindi risonar l'isola tutta
S'ode di pentolacce e di fischiate.
Come di carneval, quando in bautta
Ed in maschera vanno le brigate,
Che in larga piazza la gente ridutta,
In veggendole fàlle le risate;
Così i demonj, a vederlo in quel modo,
Ridevan fra di loro sodo sodo.
54
55
56
Incalza il prete la bestia infernale,
E le comanda che prima d'uscire,
Gli narri, come dispiegasse l'ale
In questo lido, e chi le diè l'ardire.
Mostra ben ella avere ciò per male,
E a patto alcun non lo vorrebbe dire;
Ma Dio vuol per sua lode e per sua gloria,
Ch'egli lo dica, e ne resti memoria.
57
58
59
Il signore dell'isola e la moglie
Moriro un dì da fulmine percossi;
Talchè tutto s'empì d'affanni e doglie
Il bel paese: e qual da turbin scossi,
Gli alber che prima avean sì belle foglie,
E sì bei pomi, verdi, bianchi e rossi,
Fan paura e pietade ai riguardanti;
Tali eran di quell'isola i sembianti.
60
61
62
Le grazie, il brio e l'estrema dolcezza
Che avevano parlando, chi dir puote?
Or giunte queste a quella giovinezza
Che alla vista dell'uomo si riscuote,
E s'allegra d'aver grazia e bellezza
Per lui piacere, un perfido nipote
Del morto padre, di sfrenate voglie,
Arse d'avere l'una e l'altra in moglie.
63
64
65
Il primo giorno scorse ed il secondo;
E già, qual fior che per troppo calore
Illanguidisca, il bianco e rubicondo
Color del volto lor d'atro pallore
Si ricoperse, e non fu più giocondo.
Allora quel maligno traditore
Cercò con acque e balsami possenti
Rinvigorir le forze lor cadenti.
66
67
68
E diede a noi quest'isola in domìno.
Or tu, come entri a farci dipartire?
Qui il folletto si tacque, e a capo chino
Stiè del gigante la risposta a udire.
Ed egli: Io voglio, brutto malandrino,
Ajutato dal mio superno Sire,
Che quinci tu ti parta, e parta adesso;
Se no, ti frusto senz'altro processo.
69
70
71
Ed ecco Orlando e il sir di Montalbano,
Che quivi in ritrovare i figli loro
Segni di croce si fecer con mano:
Ma usciron presto d'affanno e martoro,
Quando essi con parlare umile e piano,
Ma colmo di grandezza e di decoro,
Disser le cose come eran passate,
E lor mostraro le lor donne amate.
72
73
74
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.
ebookmasss.com