100% found this document useful (1 vote)
11 views

PolyBase Revealed: Data Virtualization with SQL Server, Hadoop, Apache Spark, and Beyond 1st Edition Kevin Feasel instant download

The document is a comprehensive guide to PolyBase, focusing on data virtualization with SQL Server, Hadoop, and Apache Spark. It covers installation, configuration, and integration with Azure Blob Storage and Hadoop, along with performance optimization techniques like predicate pushdown. The book also addresses common integration errors and provides practical examples for effective data management.

Uploaded by

omursepil
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (1 vote)
11 views

PolyBase Revealed: Data Virtualization with SQL Server, Hadoop, Apache Spark, and Beyond 1st Edition Kevin Feasel instant download

The document is a comprehensive guide to PolyBase, focusing on data virtualization with SQL Server, Hadoop, and Apache Spark. It covers installation, configuration, and integration with Azure Blob Storage and Hadoop, along with performance optimization techniques like predicate pushdown. The book also addresses common integration errors and provides practical examples for effective data management.

Uploaded by

omursepil
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 48

PolyBase Revealed: Data Virtualization with SQL

Server, Hadoop, Apache Spark, and Beyond 1st


Edition Kevin Feasel download

https://ebookmeta.com/product/polybase-revealed-data-
virtualization-with-sql-server-hadoop-apache-spark-and-
beyond-1st-edition-kevin-feasel/

Download more ebook from https://ebookmeta.com


We believe these products will be a great fit for you. Click
the link to download now, or visit ebookmeta.com
to discover even more!

Beginning Apache Spark 3: With DataFrame, Spark SQL,


Structured Streaming, and Spark Machine Learning
Library Hien Luu

https://ebookmeta.com/product/beginning-apache-spark-3-with-
dataframe-spark-sql-structured-streaming-and-spark-machine-
learning-library-hien-luu/

Cambridge IGCSE and O Level History Workbook 2C - Depth


Study: the United States, 1919-41 2nd Edition Benjamin
Harrison

https://ebookmeta.com/product/cambridge-igcse-and-o-level-
history-workbook-2c-depth-study-the-united-states-1919-41-2nd-
edition-benjamin-harrison/

SQL Server 2022 Revealed: A Hybrid Data Platform


Powered by Security, Performance, and Availability 1st
Edition Bob Ward

https://ebookmeta.com/product/sql-server-2022-revealed-a-hybrid-
data-platform-powered-by-security-performance-and-
availability-1st-edition-bob-ward/

DK Eyewitness Jerusalem Israel and the Palestinian


Territories Travel Guide Dk Eyewitness

https://ebookmeta.com/product/dk-eyewitness-jerusalem-israel-and-
the-palestinian-territories-travel-guide-dk-eyewitness/
Populist Rhetorics Case Studies and a Minimalist
Definition 1st Edition Christian Kock

https://ebookmeta.com/product/populist-rhetorics-case-studies-
and-a-minimalist-definition-1st-edition-christian-kock/

The Professionalisation of African Medicine African


Seminars Scholarship from the International African
Institute Volume 5 1st Edition Murray Last (Editor)

https://ebookmeta.com/product/the-professionalisation-of-african-
medicine-african-seminars-scholarship-from-the-international-
african-institute-volume-5-1st-edition-murray-last-editor/

Cisco ACI: Zero to Hero: A Comprehensive Guide to Cisco


ACI Design, Implementation, Operation, and
Troubleshooting 1st Edition Jan Janovic

https://ebookmeta.com/product/cisco-aci-zero-to-hero-a-
comprehensive-guide-to-cisco-aci-design-implementation-operation-
and-troubleshooting-1st-edition-jan-janovic/

Chase the Darkness 1st Edition J D Tyler

https://ebookmeta.com/product/chase-the-darkness-1st-edition-j-d-
tyler/

A Modern Introduction to Neutrino Physics Frank F.


Deppisch

https://ebookmeta.com/product/a-modern-introduction-to-neutrino-
physics-frank-f-deppisch/
A Most Unlikely Hero 07 1st Edition Brandon Varnell

https://ebookmeta.com/product/a-most-unlikely-hero-07-1st-
edition-brandon-varnell/
PolyBase
Revealed
Data Virtualization with SQL Server,
Hadoop, Apache Spark, and Beyond

Kevin Feasel
PolyBase Revealed
Data Virtualization with SQL Server,
Hadoop, Apache Spark, and Beyond

Kevin Feasel
PolyBase Revealed: Data Virtualization with SQL Server, Hadoop, Apache Spark,
and Beyond
Kevin Feasel
Durham, NC, USA

ISBN-13 (pbk): 978-1-4842-5460-8 ISBN-13 (electronic): 978-1-4842-5461-5


https://doi.org/10.1007/978-1-4842-5461-5

Copyright © 2020 by Kevin Feasel


This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the
trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the
material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Jonathan Gennick
Development Editor: Laura Berendson
Coordinating Editor: Jill Balzano
Cover image designed by Freepik (www.freepik.com)
Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street,
6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-
sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member
(owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a
Delaware corporation.
For information on translations, please e-mail rights@apress.com, or visit http://www.apress.com/
rights-permissions.
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and
licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales
web page at http://www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is available to
readers on GitHub via the book’s product page, located at www.apress.com/9781484254608. For more
detailed information, please visit http://www.apress.com/source-code.
Printed on acid-free paper
To my wife Jialin. It turns out I actually was doing something
all those hours cooped up in the office.
Table of Contents
About the Author����������������������������������������������������������������������������������������������������� xi

About the Technical Reviewer������������������������������������������������������������������������������� xiii

Acknowledgments���������������������������������������������������������������������������������������������������xv

Introduction�����������������������������������������������������������������������������������������������������������xvii

Chapter 1: Installing and Configuring PolyBase������������������������������������������������������� 1


Choose the Form of Your PolyBase����������������������������������������������������������������������������������������������� 1
Installing PolyBase Standalone—Windows���������������������������������������������������������������������������������� 5
Installing PolyBase Scale-Out Group������������������������������������������������������������������������������������������� 13
Building a Configuration File������������������������������������������������������������������������������������������������� 14
Installing Without a GUI��������������������������������������������������������������������������������������������������������� 16
Installing PolyBase Standalone—Linux�������������������������������������������������������������������������������������� 17
Configuring PolyBase������������������������������������������������������������������������������������������������������������������ 21
Configuring a Client��������������������������������������������������������������������������������������������������������������� 21
Enable PolyBase�������������������������������������������������������������������������������������������������������������������� 23
Conclusion���������������������������������������������������������������������������������������������������������������������������������� 31

Chapter 2: Connecting to Azure Blob Storage�������������������������������������������������������� 33


Making Preparations in Azure����������������������������������������������������������������������������������������������������� 33
Create a Storage Account������������������������������������������������������������������������������������������������������ 34
Upload Data��������������������������������������������������������������������������������������������������������������������������� 39
Building a Link���������������������������������������������������������������������������������������������������������������������������� 41
Credentials���������������������������������������������������������������������������������������������������������������������������� 42
External Data Sources����������������������������������������������������������������������������������������������������������� 43
External File Formats������������������������������������������������������������������������������������������������������������ 45
External Tables���������������������������������������������������������������������������������������������������������������������� 47

v
Table of Contents

Querying External Data��������������������������������������������������������������������������������������������������������������� 51


Inserting into External Tables������������������������������������������������������������������������������������������������������ 55
PolyBase Data Insertion Considerations�������������������������������������������������������������������������������� 61
Conclusion���������������������������������������������������������������������������������������������������������������������������������� 62

Chapter 3: Connecting to Hadoop��������������������������������������������������������������������������� 63


Hadoop Prerequisites������������������������������������������������������������������������������������������������������������������ 63
Preparing Files in HDFS��������������������������������������������������������������������������������������������������������� 64
Gather Configuration Settings����������������������������������������������������������������������������������������������� 68
Configuring SQL Server��������������������������������������������������������������������������������������������������������������� 70
Update PolyBase Configuration Files������������������������������������������������������������������������������������� 72
External PolyBase Objects for Hadoop���������������������������������������������������������������������������������������� 73
Credentials���������������������������������������������������������������������������������������������������������������������������� 73
External Data Sources����������������������������������������������������������������������������������������������������������� 73
External File Formats������������������������������������������������������������������������������������������������������������ 75
Querying Data in Hadoop������������������������������������������������������������������������������������������������������������ 82
Row Counts with Police Incident Data����������������������������������������������������������������������������������� 82
Newlines and Quotes with Fire Incident Data����������������������������������������������������������������������� 86
Going Faster with Parking Violations Data���������������������������������������������������������������������������� 88
Inserting Data into Hadoop��������������������������������������������������������������������������������������������������������� 92
Conclusion���������������������������������������������������������������������������������������������������������������������������������� 93

Chapter 4: Using Predicate Pushdown to Enhance Query Performance����������������� 95


The Importance of Predicate Pushdown������������������������������������������������������������������������������������� 95
Predicate Pushdown in PolyBase������������������������������������������������������������������������������������������ 97
Diving into Predicate Pushdown����������������������������������������������������������������������������������������������� 100
Packet Capture Without Predicate Pushdown��������������������������������������������������������������������� 101
Packet Capture with Predicate Pushdown�������������������������������������������������������������������������� 105
When Predicate Pushdown Makes Sense��������������������������������������������������������������������������������� 110
Small Data: Raleigh Police Incidents����������������������������������������������������������������������������������� 110
Bigger Data: New York City Parking Violations�������������������������������������������������������������������� 111
Limitations in Pushdown-Eligible Predicates���������������������������������������������������������������������� 112

vi
Table of Contents

Limitations on Pushdown with Complex Filters������������������������������������������������������������������ 117


MapReduce and Pushdown in Summary����������������������������������������������������������������������������� 124
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 125

Chapter 5: Common Hadoop and Blob Storage Integration Errors����������������������� 127


Finding the Real Logger������������������������������������������������������������������������������������������������������������ 128
PolyBase Log Files��������������������������������������������������������������������������������������������������������������� 128
Hadoop Log Files����������������������������������������������������������������������������������������������������������������� 131
Configuration Issues����������������������������������������������������������������������������������������������������������������� 135
SQL Server Configuration���������������������������������������������������������������������������������������������������� 135
Check External Resources��������������������������������������������������������������������������������������������������� 136
Check SQL Server Configuration Files��������������������������������������������������������������������������������� 137
Hadoop-Side Configuration������������������������������������������������������������������������������������������������� 138
PolyBase and Dockerized Data Nodes��������������������������������������������������������������������������������� 139
Data Issues������������������������������������������������������������������������������������������������������������������������������� 141
Structural Mismatch������������������������������������������������������������������������������������������������������������ 142
Unsupported Characters or Formats����������������������������������������������������������������������������������� 142
PolyBase Data Limitations��������������������������������������������������������������������������������������������������� 143
Curate Your Data������������������������������������������������������������������������������������������������������������������ 143
Predicate Pushdown Failure����������������������������������������������������������������������������������������������������� 143
Errors with Scale-Out Groups��������������������������������������������������������������������������������������������������� 147
Diagnosing Scale-Out Group Problems������������������������������������������������������������������������������� 147
Differing Configuration Settings������������������������������������������������������������������������������������������ 148
Patch All Nodes Consistently����������������������������������������������������������������������������������������������� 149
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 149

Chapter 6: Integrating with SQL Server���������������������������������������������������������������� 151


Meet the New Boss������������������������������������������������������������������������������������������������������������������� 152
Creating an External Data Source��������������������������������������������������������������������������������������� 152
Creating an External Table��������������������������������������������������������������������������������������������������� 154
PolyBase vs. Linked Servers����������������������������������������������������������������������������������������������������� 156
Contrasting PolyBase and Linked Servers��������������������������������������������������������������������������� 157

vii
Table of Contents

Testing a Large Table���������������������������������������������������������������������������������������������������������������� 159


Top Ten Rows����������������������������������������������������������������������������������������������������������������������� 162
Finding Ohio Plates�������������������������������������������������������������������������������������������������������������� 163
Joining Local and Remote Tables���������������������������������������������������������������������������������������� 165
Adding an Index������������������������������������������������������������������������������������������������������������������� 167
Querying Big States������������������������������������������������������������������������������������������������������������� 171
Testing a Very Large Table�������������������������������������������������������������������������������������������������������� 174
Additional PolyBase Notes�������������������������������������������������������������������������������������������������������� 176
PolyBase Supports Views���������������������������������������������������������������������������������������������������� 176
Chained External Tables Not Supported������������������������������������������������������������������������������ 178
External Table Security�������������������������������������������������������������������������������������������������������� 178
No Constraints on External Tables��������������������������������������������������������������������������������������� 181
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 182

Chapter 7: Built-In Integrations: Cosmos DB, Oracle, and More��������������������������� 183


Integration with Cosmos DB����������������������������������������������������������������������������������������������������� 183
Prepare Data in Cosmos DB������������������������������������������������������������������������������������������������ 184
Create a Database Scoped Credential��������������������������������������������������������������������������������� 190
Create an External Data Source������������������������������������������������������������������������������������������� 191
Create an External Table������������������������������������������������������������������������������������������������������ 192
Querying Cosmos DB����������������������������������������������������������������������������������������������������������� 195
Considerations When Using Cosmos DB������������������������������������������������������������������������������ 197
Integration with Oracle������������������������������������������������������������������������������������������������������������� 200
Common Oracle Errors�������������������������������������������������������������������������������������������������������� 203
Integration with Teradata���������������������������������������������������������������������������������������������������������� 203
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 204

Chapter 8: Integrating via ODBC��������������������������������������������������������������������������� 205


The Basic Flow�������������������������������������������������������������������������������������������������������������������������� 205
Integration with Apache Spark�������������������������������������������������������������������������������������������������� 206
Obtain and Install a Valid Driver������������������������������������������������������������������������������������������ 206
Create a Data Source Name������������������������������������������������������������������������������������������������ 208

viii
Table of Contents

Create a Database Scoped Credential��������������������������������������������������������������������������������� 210


Create an External Data Source������������������������������������������������������������������������������������������� 211
Create an External Table, Take 1������������������������������������������������������������������������������������������ 212
Create an External Table, Take 2������������������������������������������������������������������������������������������ 214
Integration with Apache Hive���������������������������������������������������������������������������������������������������� 217
Obtain and Install a Valid Driver������������������������������������������������������������������������������������������ 217
Create a Data Source Name������������������������������������������������������������������������������������������������ 218
Create a Database Scoped Credential��������������������������������������������������������������������������������� 221
Create an External Data Source������������������������������������������������������������������������������������������� 221
Create an External Table������������������������������������������������������������������������������������������������������ 222
Integration with Microsoft Excel����������������������������������������������������������������������������������������������� 223
Create an External Data Source������������������������������������������������������������������������������������������� 225
Create an External Table, Take 1������������������������������������������������������������������������������������������ 225
Create an External Table, Take 2������������������������������������������������������������������������������������������ 228
An Extended Note on Driver Support���������������������������������������������������������������������������������������� 230
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 231

Chapter 9: PolyBase in Azure Synapse Analytics������������������������������������������������� 233


A Different Breed of PolyBase��������������������������������������������������������������������������������������������������� 233
Creating an Azure Synapse Analytics SQL Pool������������������������������������������������������������������������� 236
Loading Data into Azure Synapse Analytics������������������������������������������������������������������������������ 239
PolyBase Differences in Action������������������������������������������������������������������������������������������������� 242
CTAS and Command Progress��������������������������������������������������������������������������������������������� 242
Rejected Row Location�������������������������������������������������������������������������������������������������������� 244
Defining the First Row��������������������������������������������������������������������������������������������������������� 247
The CETAS Statement���������������������������������������������������������������������������������������������������������� 248
Additional Functionality������������������������������������������������������������������������������������������������������� 249
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 249

ix
Table of Contents

Chapter 10: Examining PolyBase via Dynamic Management Views��������������������� 251


Dynamic Management Views���������������������������������������������������������������������������������������������������� 251
PolyBase Dynamic Management Views������������������������������������������������������������������������������������ 252
Metadata Resources������������������������������������������������������������������������������������������������������������ 252
Service and Node Resources����������������������������������������������������������������������������������������������� 254
Resources for Troubleshooting Queries������������������������������������������������������������������������������� 258
Understanding PolyBase Queries Through Dynamic Management Views��������������������������������� 261
Blob Storage������������������������������������������������������������������������������������������������������������������������ 261
Hadoop with Predicate Pushdown��������������������������������������������������������������������������������������� 265
SQL Server��������������������������������������������������������������������������������������������������������������������������� 268
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 271

Chapter 11: Query Tuning with Statistics and Execution Plans���������������������������� 273
Statistics in SQL Server������������������������������������������������������������������������������������������������������������ 273
Statistics on External Tables������������������������������������������������������������������������������������������������ 276
Managing External Statistics����������������������������������������������������������������������������������������������� 277
The Performance Impact of Statistics��������������������������������������������������������������������������������� 280
Tuning Queries with Execution Plans���������������������������������������������������������������������������������������� 283
Reviewing an Execution Plan���������������������������������������������������������������������������������������������� 283
Reviewing MapReduce Queries������������������������������������������������������������������������������������������� 287
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 288

Chapter 12: PolyBase in Practice������������������������������������������������������������������������� 289


Cold Storage����������������������������������������������������������������������������������������������������������������������������� 289
Alternate Approaches���������������������������������������������������������������������������������������������������������� 296
Replacing ETL Jobs with ELT����������������������������������������������������������������������������������������������� 296
Third-Party Applications and External Tables���������������������������������������������������������������������� 300
Generalizing PolyBase Scenarios���������������������������������������������������������������������������������������������� 303
Conclusion�������������������������������������������������������������������������������������������������������������������������������� 304

Index��������������������������������������������������������������������������������������������������������������������� 305

x
About the Author
Kevin Feasel is a Microsoft Data Platform MVP and CTO at
Envizage, where he specializes in T-SQL and R development,
forcing Spark clusters to do his bidding, fighting with Kafka,
and pulling rabbits out of hats on demand. He is the lead
curator at Curated SQL (https://curatedsql.com).
A resident of Durham, North Carolina, Kevin can be found
cycling the trails along the Triangle whenever the weather is
nice enough.

xi
About the Technical Reviewer
Ike Ellis has over 18 years of experience in data engineering.
He’s a current Microsoft MVP. He is the General Manager of
Data & AI for Solliance. He is a partner in Crafting Bytes,
a San Diego software studio and Data Engineering group.
He is an author who has written several books on Microsoft
data and Azure topics. Ike has spoken at conferences around
the world including PASS Summit, SQLBits, DevIntersection,
TechEd, TechEd Europe, and SQL in the City. He’ll be
speaking at Craft in Romania in 2020. For more information,
see www.ikeellis.com or follow him on Twitter at @ike_ellis.

xiii
Acknowledgments
This book is the product of many hours of research, rants, and head-scratching
moments. Most of the head-scratching was mine, and I could not have written this book
without the help of some very smart people. First, I would like to thank several Microsoft
employees. Murshed Zaman gets top billing here for starting me on my PolyBase
journey and helping solve my initial problems. James Rowland-Jones is next—James
patiently explained various PolyBase details to me over and over, until I finally got
them. Eric Burgess, Nathan Schoenack, and Suresh Kandoth at Microsoft CSS helped
me understand common pain points and shaped the direction of several chapters,
particularly Chapter 5. Finally, Jasraj Dange and Weiyun Huang helped connect the
dots on PolyBase V2 functionality and left me excited about the future of the technology
beyond SQL Server 2019.
I am grateful to Bill Preachuk and Scott Shaw at Cloudera (and previously, when they
were at Hortonworks) for helping me through several land mines early on when working
with Hadoop. I’ll never forget going back and forth between Murshed at CAT and Bill
and Scott at the Hortonworks booth at PASS Summit trying to get PolyBase and the
Hortonworks Data Platform sandbox working at the same time on my laptop.
In addition to these two companies, I am heavily indebted to several people in the
community. Ike Ellis is the best technical editor I could have asked for, at least until he
calls in the favor. Gerhard Brueckl was great to bounce ideas off of and provided me with
insights I never would have discovered otherwise. Jason Horner helped me test things I
thought I knew about the product and made this book better as a result. Hasan Savran
taught me most of what I know about Cosmos DB, though don’t hold my limitations
against him. There are plenty of others who played a role in this book, and I thank you all
for it.
As is natural in these works, any errors which remain are mine and mine alone,
although I will probably try to blame gremlins.

xv
Introduction
This is an exciting time to be a data platform professional. Over the past decade, we have
seen a proliferation of data platform technologies, all trying to solve the critical problem
of our era: collecting, storing, managing, and querying ever-increasing amounts of data.
To solve this problem, we have seen the rise of technologies like Apache Hadoop, Apache
Spark, Google BigTable, Amazon Redshift, Microsoft Azure Synapse Analytics, and plenty
more. In the meantime, the hard core of the data platform space—relational databases—
has not ossified. Each new edition of SQL Server, Oracle, and PostgreSQL has new
features and the ability to handle even more data. As great as these relational database
platform products are, however, they do not fit every use case. Where there are gaps,
other products fill the void. This leaves data platform developers at most enterprises—
even companies of moderate size—juggling data between several systems.
Historically, the way we managed this juggling act was to learn a separate language
for each platform: T-SQL for SQL Server; PL/SQL for Oracle; HiveQL for Apache Hive;
Spark SQL, Scala, or Python for Apache Spark; and so on. The common adoption of SQL
as a general interface (sometimes in spite of product developers’ wishes) has simplified
the task, but each product has its own dialect of ANSI SQL, and it can be difficult to
remember which operators and functions exist in one database vs. another.
Furthermore, the most common task I see in this space is some variant of Extract-­
Transform-­Load (ETL): moving data from one system to another, sometimes reshaping
it along the way, in order to combine the products of two separate systems. We spend so
much time moving data within systems, going from write-heavy Online Transactional
Processing (OLTP systems) into reporting-friendly Online Analytical Processing (OLAP)
databases. Add different data platform technologies and the problem grows even further:
now we need to combine that general ledger data from Oracle, device statuses from
Spark, device metadata from SQL Server, customer data entry from Cosmos DB, and
historical rollups from Teradata in order to build a neural network which will solve all of
our problems. The traditional approach has been to use purpose-built ETL tools like SQL
Server Integration Services or Informatica, or to write custom code in a programming
language of choice. These techniques work, but they are relatively effort-heavy, particularly
in maintaining this separate ETL process as the source systems evolve over time.

xvii
Introduction

A modern take on the classic problem of ETL is data virtualization: making this data
appear to come from one source system while under the covers defining links to where
the data really lives. An end user or analyst can read this data using one SQL dialect and
join together structured data sets from different systems without needing to know the
provenance of each data set and without waiting for database developers to build in the
plumbing needed to move data from one system to the next. This simplifies greatly the
analyst’s life and is one of the key selling points of Microsoft’s PolyBase technology.
PolyBase has been around since 2010 but came to the general public in SQL Server
2016. Its purpose was to integrate SQL Server with Hadoop by allowing us to run
MapReduce jobs against a remote Hadoop cluster, bringing the results back into SQL
Server and thus reducing the computational burden on our relatively more expensive
SQL Server instances. Now, with SQL Server 2019, PolyBase has grown and adapted to
this era of data virtualization. As you will see throughout this book, PolyBase gives us
the ability to integrate with a variety of source systems. In the book, we will connect to
a Hadoop cluster, Azure Blob Storage, other SQL Server instances, an Oracle database,
Cosmos DB, an Apache Spark cluster, Apache Hive tables, and even Microsoft Excel! This
leaves out the wide variety of other data sources, such as Teradata, MongoDB, DB2, and
much more. The best part of it is that our developers need only one language for all of
this: T-SQL.
PolyBase is no panacea, and there are certainly trade-offs compared to storing all
data natively in one source system, particularly around performance. If you do, however,
have existing, disparate systems which need to interact, PolyBase has a few tricks up its
sleeves to make those integrations easier.
This book is intended for database developers, database administrators, and
architects looking to solve multisystem integration problems. My key assumption
throughout this book is that you are already familiar with the T-SQL language but
might be less familiar with different data platform technologies such as Hadoop, Spark,
or Cosmos DB. Naturally, having more experience with these other data platform
technologies will help considerably when dealing with the headaches which come when
trying to interconnect disparate systems.
My intent in this book is as much narrative as reference, meaning that the best way
to read the book is in chapter order. Even if you do not make use of a particular data
platform technology, there can be key components in the chapter which apply to other
technologies. In particularly important cases, I will note when this is the case so you do
not miss out on critical information.

xviii
Exploring the Variety of Random
Documents with Different Content
caution, but if no such habits are the result, danger ceases to excite
this emotion, and a man becomes at once fearless and careless. So
with sympathy in the sufferings of others; if no habits of benevolent
efforts to relieve are induced, that sensibility diminishes, and men
become at once unsympathising, hard and cruel. So it is with
shame; if it does not lead to habits of honor and duty, the
susceptibility continually diminishes. And so it is with remorse; if
habits of rectitude are not induced by its emotions, the conscience
becomes “seared as with hot iron.”

But the most deteriorating effect of wrong action is seen in regard to


that fundamental point of the mental constitution which makes it a
source of happiness to [pg 175] be the cause of happiness to others.
It is a universal fact that the tendency of disagreeable emotions is to
lead to the infliction of pain on others. This propensity to inflict pain
on whoever is the cause of pain, when regulated by the rules of
rectitude, is the source of justice in the family and state, and leads
only to good. But when it is indulged and unregulated, it is the most
fearful feature in our mental constitution. The records of history
exhibit many monsters of our race, whose mental constitution has
become so disordered by habits of fatal indulgence, that all love of
happiness-making for others seems destroyed, and the baleful
pleasure of tormenting becomes a ruling passion.

Another feature of our mental conformation which directly bears on


this subject, is the fact, that all those good qualities and benevolent
acts which naturally tend to please and awaken the desire of good to
others, may become sources of pain and ill-will. This is the case
when the lovely and benevolent traits of other minds are contrasted
with opposite traits in self. Thus it is that the selfish, cruel and
malignant hate and are powerfully repelled from the generous, just
and virtuous, while the good as instinctively fly from the wicked.

The natural result of these features in the nature of mind, is a


continual tendency toward a separation of the good and the bad, the
righteous and the wicked.
According to the teachings of experience, a mind that forms habits
of selfishness and sin is constantly tending to a deterioration of its
nature in all directions. And the course of obedience to the grand
law of self-sacrifice for the best good of all, becomes more [pg 176]
and more difficult and improbable. As the natural result the good are
more and more attracted toward each other, and the bad are more
and more repelled.

These tendencies, so plainly exhibited here, reasoning from


experience, we infer are to continue after death, until the final result
must be the entire exclusion of the evil from the good, whenever
power exists to compel the separation. This power, all must feel is
held and will be exercised by the Author of all minds, whose great
plan, so far as reason teaches, can be carried to perfection only by
such a consummation.

One point in the history of our race has a mournful pertinence to this
question. We find that the improvement and the safety of the great
commonwealth is always, more or less, promoted by the ruin of
individuals. Multitudes are deterred from evil courses by the
miserable end of those who pursue them; so that the good are often
preserved by the destruction of the bad.

So, too, we find exhibitions of the fact that minds are utterly ruined,
and ruined for ever, so far as we can perceive. The man who has
stultified his intellect, ruined his health, seared his conscience, and
blunted all his generous and benevolent sensibilities by a course of
debauchery, cruelty and crime, is a wreck as total and irretrievable,
so far as we can see, as a watch whose springs and pivots are
crushed beneath the hammer, or a human body whose every
lineament is effaced beneath the rushing locomotive train.

The common language of life expresses such mental facts in


precisely the same terms as are applied to physical catastrophes.
Thus, a man who is given [pg 177] up to debauchery, intemperance
and crime, is said to be a “total wreck”—“entirely
destroyed,”—“utterly ruined.”

Add to this the teaching of experience, that when men are bad, the
increase of blessings only increases indulgence and crime. At the
same time punishment does not tend to reformation. The more men
suffer for their folly and guilt, the more hardened they become. The
victims of licentiousness and intemperance, though they suffer such
miseries, have ever been regarded as the farthest removed from the
probabilities of reformation.

Add to all this, the deductions of reason as to the moral nature of


the Creator and Governor of all minds. He has power to separate the
good and bad; his great design, of which we here see only the
tendencies, makes it indispensable to the perfect happiness of the
good that they be separated from the bad—a perfectly happy
commonwealth can not be attained where the bad form a part—
while the sense of justice exists in God on a scale far above ours,
demanding added penalties for the known and willful destruction of
happiness. He, like his children on earth, feels that craving for
retributive justice, which can never rest till the guilty and
remorseless monster receives the just recompense for his cruelty
and crimes.

These teachings of reason and experience lead to the conclusion,


not only that there is to be a grand consummation in which all sin
and suffering shall be ended in a perfected commonwealth, but also
to the conclusion that those excluded from this community of the
good are to continue their existence in sin and its natural results for
ever.

[pg 178]
That any portion, either of matter or mind, is to be annihilated, can
not be inferred from any past experience. All that we can learn are
the laws of perpetual succession and change. One single fact of
annihilation has never yet been made known to man by any process
of reasoning, or any recorded experience.

There is another question in reference to this awful subject, which is


of deepest interest. Although the deductions of reason lead to the
doctrine of the eventual separation of mankind into two distinct
communities, the good and the evil, what are its teachings as to the
immediate state of each individual soul after the event of death?

Here, as before, we have only the nature and past history of mind,
from which the future is to be deduced. In this world we have found
the changes in the character of individuals and of communities to
proceed by slow and imperceptible movement. We have nothing in
the past to lead to the belief that this slow process of discipline,
culture and change may not proceed on for ages. As in this life,
multitudes have the impress and direction of character given in early
life, so that the first few years determine all their future history in
this world, so the career of this short life may fix the future through
eternal years. And yet the process of change to the full
consummation of character may involve ages.

In studying the works of the Creator, we find that every thing goes
forward on a system of developments. Nothing comes into being in
full perfection, and unless there is an interruption of the natural
tendencies of things, every thing reaches its full and perfected state
before its existence ends. And the nobler, larger, [pg 179] and
grander the existence, the slower it proceeds to its consummated
perfection. The oak and the palm demand centuries ere they reach
their perfected prime. The highest grades of animal life are slowest
in gaining their full development. The horse, the elephant, and the
camel, are going forward to perfection for years after the feebler
tribes that started with them have perfected and perished.

Guided, then, by the analogies of experience, we should infer that


mind, the noblest work of its Creator's hand—mind, that begins its
career in such low and feeble development, is not to form the
mournful exception to the general rule.

On the contrary, we infer from all past experience, both of matter


and mind, that the soul, when it lays aside its outer covering,
proceeds onward in its career of development. And if its period of
progressive development is proportioned to its relative value in
comparison with all other created things, the fleeting years of this
life in relation to the ages previous to its prime, may be but as the
first days of puling infancy to the whole career of manhood.

But this subject is imperfectly treated, if we neglect to consider the


fact, that the soul, so far as we can perceive, is disembodied at
death. We have perfect evidence, that the material part is destroyed,
as to its organized existence. We have the same sort of evidence
that the soul continues to exist, and will continue to exist, as we
have that the sun exists when all evidence of sight ceases. But what
is the experience of a disembodied spirit, we have no means of
learning. It may be that its powers of knowledge and action are
greatly increased, when freed from its earthly prison. [pg 180] If this
be so, the experience of this life leads to the inference that its
dangers and temptations are increased in exact proportion. Increase
of civilization is only increase in sources of knowledge and
enjoyment, and each addition brings new temptations, new rules,
and the need of new penalties. It may be the same in the future life.

We can suppose the body a veil to hide our mind from another, and
that death makes every soul “open and naked,” in all its thoughts
and feelings, to every other disembodied spirit. What would be the
effect of such a revelation, no one could say. But we should fear
rather than hope.

If men are exasperated by words that exhibit only a portion of the


scorn, contempt, and disgust felt toward the base and mean, not
only by the pure and good, but by the wicked themselves, such a full
revelation of all minds to all minds presents a theme for awful
forebodings to the guilty. And even the purest might tremble to
encounter such an ordeal. But over such terrific conjectures rest the
darkness and silence of the grave.

The following, then, are the deductions of reason and experience as


to the future condition of our race after death.

The soul, at the dissolution of the body, remains unchanged in its


tastes, habits and character. The tendencies indicated in this life are
continued indefinitely, and eventually will result in the separation of
the good and the bad into two separate communities, the one, being
obedient to all the laws of God, will be for ever and perfectly happy,
and the other are to reap the natural results of disobedience, and
whatever [pg 181] added penalties the best good of the universe
may demand.

The final consummation in which this separation will be achieved,


may be at the distance of ages, and in the meantime all those minds
that have passed, or will pass from this life, are in the same process
of culture, discipline, and upward or downward progress, which
exists in this life. Whether these advantages and temptations will be
greater or less in the disembodied state, we have no data for
inference or conjecture.

The conduct and character formed in this life will have an abiding
influence on the character and happiness of every mind through
eternal ages.10
Chapter XXIX. What Must We Do To Be
Saved?

We have considered the risks and dangers of the future state, as


taught by reason and experience, and also as the foundation of a
true standard of morality. We have seen that the true mode of
escape from these dangers is the formation of a truly virtuous
character, or in other words, it is making it our chief end to obey all
the laws of God.

The next question is, what are the teachings of reason and
experience as to the most successful modes of securing true virtue,
or voluntary obedience to all the laws of God?

This brings up the inquiry as to the causes of voluntary [pg 182]


action, and of the power which one mind has of securing right or
wrong volitions in another.

In a previous chapter was pointed out the distinction to be


recognized between the producing cause and the occasional causes
of volition.

Mind itself is the only producing cause of its own volitions. Excited
desires, and those objects which excite desire, are the occasional
causes of choice.

The question is, in what sense can any being be the cause of
virtuous actions, or virtuous character, in another mind?
Here we must recur to the fact that the Creator, as the author of all
minds, and of all the things that excite desire, is the cause, in one
sense, of all the volitions and of all the characters of all finite minds.
It is in this sense that, in the Bible, the Jehovah of the Old
Testament says, “I make peace and create evil.” No other being but
the Creator can be regarded as the cause of volitions in this sense,
viz., as the author of all minds and their circumstances of
temptation.

There is a second sense in which the Creator is never the cause of


sinful action in any mind. It is this: creating or modifying our
susceptibilities, or arranging temptations with the design or intention
of producing sinful action. This is established by proving, that the
chief end of God is to make the most possible happiness, and that
sin is the needless destruction of happiness, resulting from
disobedience to the laws of God.

The only sense, then, in which God can be called the author or
cause of sinful volitions in the minds of his creatures, is the fact that
he is the author of all created minds and of their circumstances of
temptation.

[pg 183]
In regard to man, there are only two conceivable modes, in which he
can be the cause of sinful or virtuous character in other minds.

The first mode is so to combine circumstances of temptation as to


affect the most excitable and powerful sensibilities, or to remove
those objects and influences that sustain moral principle, or by a
long course of training, to form habits and induce principles. The
combinations of motive influences that one mind can thus bring to
bear on another, as temptations to right or wrong action, are almost
infinite.

Another mode is by changing the constitutional susceptibilities. This


can sometimes be effected to a certain degree by education, and the
formation of habits. It can be still more directly effected through the
physical organization. For example, a child may be trained to use
coffee, tea, alcohol, or tobacco, till the nervous system is shattered,
and then a placid temper becomes excitable, an active nature
becomes indolent, and multitudes of other disastrous changes are
the result.

When these two modes are employed with the design to induce
wrong action, then men are blameable causes of sinful action and
character in their fellow men. God, as above shown, never thus
causes sin. When these modes are employed with the intention to
induce virtuous actions and character, then both God and man are
causes of right moral action in mankind.

Thus, it appears, that in the formation of virtuous character and


habits, God, educators and self are the three combining causes,
each being indispensable to the result, and thus each dependent on
the others. God decides the nature and combinations of our
susceptibilities [pg 184] and our circumstances of temptation. The
educators of mind also modify the susceptibilities, and regulate the
temptations. Self, as the producing cause of volition, decides the
nature of our own volitions, and thus also coöperates to regulate
circumstances of temptation.

The attainment of virtuous character, therefore, depends conjointly


on God, man and self. It has been shown that God invariably does
the best he can to secure the most perfect action possible in all
minds.

The blamable causes of all failure in right and virtuous action are self
and the finite educators of self. The unblamable causes are God,
educators and self, so far as they are faithful in doing all they can to
educate aright.

With these preliminary considerations, we proceed in the inquiry as


to those modes which in past experience have been found most
successful in securing virtuous character, or voluntary obedience to
the laws of God.

The first cause of right moral action is a knowledge of and faith in


the physical, social, intellectual and moral laws of God. It is
impossible, in the nature of things, that a new-created mind should
be possessed of such knowledge and faith. All that is possible, so far
as we can learn by reason and experience, is that there should be a
slow and gradual development not only of each individual mind, but
of the whole race, as each generation, in turn, receives by
instruction the experience of the one previous, and transmits it with
its own experience to a succeeding generation.

The next thing that has been found efficacious in forming virtuous
character is the formation of uniform habits of obedience to parental
rule, in the early periods [pg 185] of existence. To secure this,
invariable steadiness in government has been found indispensable. If
a child finds that sometimes he is to obey and sometimes he is not,
there is always a temptation to struggle against law. But if a parent's
laws, rewards and penalties are as steady and sure as those of God,
in due time the child submits as cheerfully to the domestic rules and
commands, as he does to the laws of nature. He is no more tempted
to contest parental commands than he is to attempt to stop the flow
of a river or the falling of rain. In this way a habit of submission to
law is generated, which makes all the future discipline and training
of life comparatively easy. A child learns cheerfully to obey a
heavenly Father, just in proportion as he thus obeys his earthly
parents.

The next thing taught by experience is that children should be


instructed as fast as possible in the reasonableness and benevolence
of all the laws they are required to obey. Obedience is made easy
and sure just in proportion as a child is made to perceive, that such
obedience is best for himself and best for all concerned.
The next thing which experience has shown to be most effective in
securing obedience to law, is love on the part of the educator, and
corresponding love in return from the child. To gain the love of a
child an educator must exhibit all lovable traits, and confer benefits,
so as to call forth at once admiration, gratitude and affection. This
renders it easy to the child to conform to the rules and wishes of
one so beloved.

Sympathy with a child in all its trials and in all its enjoyments, still
further increases this power of another mind in right guidance.

[pg 186]
This sympathetic influence is greatly increased by the power of a
virtuous example—especially if this example is exhibited by a
beloved friend and benefactor, who would be gratified by thus
guiding a dependent mind.

Another influence that tends to secure virtuous action is the bearing


of pain and hardships even when it is not voluntary. Those children
who are trained in a cold clime and on a hard soil, and who are early
trained to hardships, find it far easier to conform to rule, and to bear
sacrifices for the general good, than those whose lives have been a
course of uninterrupted ease and indulgence.

To these, add the social influences of the example and sympathy of


a surrounding community. Where all around are practicing virtuous
conduct—where all admire and praise only what is good and right—it
is far easier to secure obedience to the rules of rectitude, than
where the example and sympathy of surrounding minds are opposed
to virtue.

But the most powerful of all influences in securing virtuous action, is


the principle of love and gratitude toward some noble benefactor,
who saves from some terrible evils at the expense of great personal
suffering and sacrifices, and who seeks his reward in the pleasure of
redeeming those thus benefited, from the snares and ruin of sin.
And the greater the evils averted, and the more severe the suffering
on the part of the benefactor, the stronger the influence thus gained
to secure virtuous character and action in the one thus rescued.

These are the influences which experience has [pg 187] shown to be
most effective in securing virtuous character.

When the question is asked, “What must we do to be saved?” it may


be answered in reference to all concerned in the matter; that is to
say, “What must self do—What must our fellow-men do—What must
the Creator do, to secure obedience to his laws, and thus to save
from sin and its penalties?”

In view of the above teachings, each one for himself must seek,
first, knowledge of the laws of God, and of their rewards and
penalties as discovered by the experience of mankind. In order to do
this, each must take all means to gain true teachers, and to receive
their teachings in true faith, that is, that practical faith, which
includes the purpose of obedience. Each must cultivate the intellect,
the reason and the moral sense, in order to judge correctly in
receiving and applying the rules of rectitude; each must seek to
discover the reasonableness and benevolence of these laws, and
form habits of steady obedience; each must seek to discover and
rightly to appreciate all the good and lovable qualities of all who
institute and administer laws, from the Creator to all subordinate
rulers and governors in the domestic and civil state; each must seek
the society of those whose sympathy and example would encourage
and promote virtuous conduct; and finally, each must make
obedience to all the laws of God the chief end or ruling purpose.
These are briefly the reply to the great question in relation to self.

We are next to consider this question in relation to what men must


do to save others.

Here we are to take into account two subjects previously [pg 188]
illustrated; the first is that great law of sacrifice, by which each
individual must make his own wishes and welfare subordinate to the
higher interests of the great commonwealth; the second is the fact
that all questions of right and wrong are dependent on the risks and
dangers that threaten the commonwealth. In cases where there is
little peril or evil, each individual has little responsibility for others.
On the contrary, when all are exposed to terrific dangers and
hazards, every individual is bound to think and care as much for the
danger of each one as for his own. And just as much as the interests
of all are of more value than those of one, so much more should
each place the public welfare above that of self.

In a preceding chapter have been exhibited the risks and dangers of


our race in reference to the future life. These are such, that without
any appeal to revelation, every man of humanity and benevolence
must feel that to save his fellow-beings from such dangers should
become immediately his leading object of pursuit, his chief end.

In pursuing this as the main object of life, each individual is bound


to follow the teachings of experience as to the most successful
modes as set forth above. Each one, then, should become a teacher
of the laws of God to all who are in ignorance, to the full extent of
his power, and set forth all the motives to induce obedience; each
should strive to exhibit all those qualities and deeds which will excite
admiration, love and gratitude, in order thus to gain influence over
other minds and guide them to virtuous conduct. Each should confer
benefits and practice self-denying benevolence toward others and
thus gain still farther [pg 189] influence. Each should strive to
exhibit that example and that sympathy that are so effective in
leading others aright.

In regard to those who are the educators of the young, each must
strive to maintain that invariable steadiness in governments which is
so effective in forming virtuous habits and in rendering obedience to
the laws of God more and more easy.
Finally, it should be the aim of each to establish such a community
around all who are being trained to virtue, that every social influence
shall repress vice and encourage virtue.

Next, we are to consider the great question in reference to the


Creator. What then must God do to save our race from sin and its
miseries? What would reason and experience teach us to expect he
would do to secure obedience to his laws?

In answering this question we must again refer to the causes which


experience has shown to be most effective, for we can conceive of
no other. We have examined the evidence that the Creator has given
to each of his children such a constitution of mind and body, and
such circumstances of temptation and trial as is best on the whole,
as a part of an infinite system whose results are to develop through
eternity. At the same time it has been shown that God is limited, by
the eternal nature of things, to a course in which some evil must
exist, so that all that is requisite to his character as perfectly
benevolent, is that this evil should be reduced by him to its least
possible amount.

To suppose that God can impart at creation of each mind all the
knowledge of the millions of rules needed [pg 190] for all the
myriads of new relations, of myriads of beings through all eternity, is
to suppose an impossibility in the nature of things.

If it be maintained that the Creator is not thus limited by the nature


of things, but, as theologians teach, could make mind perfect in all
needed knowledge as in all other respects, at the first, then we have
the greater contradiction involved in the fact, that a perfectly
benevolent being chose for his children ignorance and sin in
preference to knowledge and virtue.

To say that it may be best to create minds destitute of all needed


knowledge when the want insures infinite wrong and suffering, and
when there is power to create the knowledge that would insure
perfect happiness, is simply a direct contradiction. It is saying that
less happiness may be greater than greater happiness. For by “what
is for the best” we understand “that which secures the most
happiness.” And saying that making misery where there is power to
make happiness in its place, is best, means nothing else but the
assertion above, that less happiness is greater than greater
happiness; or that less is more than most, which is a contradiction,
inconceivable and absurd, so that no mind can either comprehend or
believe it.

Now, every theologian of every school and of every sect maintains


that “God does all things for the best.” Every one who believes in a
benevolent Creator does the same. This is simply saying that God
does the best possible; that is to say, there is no power that can
make a better system than God has made, or administer it with
more wisdom or benevolence. He has chosen the best possible and
so he can not do any better.

[pg 191]
These things being granted, the teachings of experience would lead
us to suppose, still farther, that the Creator must do all that is
possible to maintain invariable steadiness of government. We can
see that this, which is so important in family government, must be
still more so in an infinite family. For this end, the natural penalties
for wrong doing, must be as invariable as the rewards for well doing.

Again, the Creator must instruct his creatures in his laws and their
rewards and penalties to the full extent of his power. That is to say,
he must provide well-trained educators of mind, as fast and as fully
as is possible in the nature of things, having in view the results of
eternal ages to guide his decisions.

Again, to secure voluntary obedience, he must add to the natural


rewards and penalties of his laws, the other class of motives which
experience has shown to be most effective. Thus, he must present
himself to his creatures as a being possessing all those qualities
which call forth the delightful emotions of admiration, reverence and
love; he must show himself as a constant benefactor, and as one
who “does not willingly afflict or grieve the children of men.” He
must manifest his love to his creatures by word as well as by deed.
He must come personally to provide for their wants and cheer them
with his care. He must show his tenderness and sympathy in their
trials and sorrows as well as in their joys. And if they are exposed to
great dangers and evils from which they can be redeemed by self-
sacrifice and suffering on his part, this highest and most effective
proof of love must be exhibited.11

To this must be added, a manifestation of his chief [pg 192]desire,


so that when love and gratitude ask, what can we do to please our
benefactor in return, the answer shall be, obey his laws, and work
and suffer for the good of all, as you see your Heavenly Parent does
for you.

Finally, he must bring around each of his creatures the powerful


social influence, not only of his own sympathy and example, but
those also of a perfect commonwealth, where all shall be perfect as
is the Father of all.

This is what we should evolve by the light of reason and experience,


as what the Creator must do to save our race. Whether he has done
all this, is a question that belongs to that system of religion which
we can gain only by revelation from God.12
Chapter XXX. How Far Reason and
Experience are Sufficient Without
Revelation.

The preceding chapters present the system of natural religion, as it


may be gained by experience and those principles of reason or
common sense with which all men are endowed.

Whether mankind ever have, or ever would, fully evolve this system
of religious belief without any aid by revelation from the Creator, is a
question which we can not readily decide—inasmuch as the claim of
Christianity is, that from the first, our race have been instructed by
revelations from God, which have been more or less preserved in
traditions and written records. [pg 193] It is certain that the
elimination of this system, by unaided humanity, is dependent on the
development of both the intellectual and moral powers, just as much
so, as the physical discoveries of Newton, Copernicus and Columbus
were dependent on the intellectual progress of the race.

In reference to the question of the necessity or importance of


revelations from the Creator, it is interesting to examine how far
those nations that have been most advanced in intellectual
development, have secured this system of common sense,
independently of the revelations contained in the Bible—revelations
which also have been more or less incorporated by Mohammed into
the Koran.

In a brief review of the pagan systems, that of Boodhism occupies


the first place, as one which has had longest and largest control over
civilized pagandom—one which has been most unimpeded by
resistance, and one which now controls one half the human race.

We have seen that the common-sense system teaches an eternally


self-existent Creator, perfect in knowledge, wisdom, power and
benevolence, administering a perfect system by laws—his chief
design being to produce the most possible happiness with the least
possible evil. It teaches also, that the right voluntary action of mind,
as a part of this system, consists in good willing toward the Creator,
toward self, and toward our fellow-beings, according to the laws of
God, so as to secure what is best for all concerned—making it
imperative that self be made subordinate to the public good. It
teaches also, that the most effective mode of securing this right
action is, first, by imparting [pg 194] a knowledge of these laws and
their sanctions, and thus influencing mind by the motives of hope
and fear; next, by the motive influences of love, gratitude, sympathy
and example, as mutually exercised by God, our fellow-men and self.
Finally, it teaches that all questions as to what is right and wrong,
are to be regulated with reference to the risks and dangers of a
future life, and not with chief reference to this life alone—and that in
this estimate the interests of self are to be made subordinate to
those of the commonwealth.

We will now notice how far the system of Boodh corresponds with
that of common sense.

This religion13 is one in which there is no intervention of any


supreme God, or any self-existent being, or any Creator; on the
contrary, all souls and all the universe exist from eternity. All souls
from eternity have gone on transmigrating from one body to
another, rising or falling in the scale of existence according to their
merit or demerit. Boodh is a general name for a divinity or god.
There have been innumerable Boodhs in different worlds and
different ages, but in this world only four. These four are beings who
have risen by merit through various transmigrations, and then
became incarnate in human bodies. At last they were annihilated,
none of them being now in existence—so that this world for
centuries has been without any God.

The last Boodh of this world was Gaudama. He passed through


innumerable transmigrations in four hundred millions of worlds, and
attained immense merit. At last, he was born into this world the son
of a king, about six hundred years before Christ. [pg 195] The
moment he was born he exclaimed, “Now am I the noblest of men;
this is the last time I shall ever be born!” He remained forty-five
years as Boodh of this world—performed all sorts of meritorious
deeds, promulgated excellent laws, and then was annihilated. Ever
since, this world has had no God, and will have none for eight
thousand years, when the next Boodh is to appear. The first three
Boodhs left no laws or sayings. Those of Gaudama, the last Boodh,
were reduced to writing A. D. 94, and these are the Bedegat, or
Bible of the Boodhists.

These teachings of Gaudama are so obligatory, that disbelief of them


is the only crime that incurs eternal punishment.

According to this system, true virtue or rewardable merit, consists in


obeying the teachings of Gaudama. These teachings relate first to
sins to be avoided. The five general laws are, not to kill, not to steal,
not to commit adultery, not to lie, and not to drink intoxicating
liquors. These are subdivided so as to include all sins of similar kinds
under each head. For example, the first law includes even the killing
of animals for food, also capital punishments and war.

Sins are divided into these three classes: first, those of the body,
such as killing, theft, fornication, etc.; those of the tongue, as
falsehood, harsh language, idle talk, etc.; and those of the mind, as
pride, covetousness, envy, heretical thoughts, etc.

These writings of Gaudama strongly denounce the evils of pride,


anger, covetousness, and all inordinate appetites. Men are exhorted
to avoid excess in perfumes, ornaments and laughter—also strong
drink, smoking, opium, night wanderings, bad company, [pg 196]
idleness, anger under abuse, flattery to benefactors, annoying jests,
and all that leads to strife.

For all such sins the most awful conceivable punishments are to
follow in a future state, and for millions of ages.

Rewardable merit is of three kinds:

1. Obedience to all the preceding precepts and prohibitions, and the


performance of all duties fairly deducible from them, such as
integrity, gentleness, lenity, forbearance, condescension, veneration
to parents and love to mankind in general.

2. Alms-giving and votive offerings. This includes feeding priests,


building temples and accommodations for priests and for travelers,
making roads, tanks and wells, planting fruit and shade trees,
feeding criminals and animals, and finally, giving alms to all classes
of men in need.

3. Prayers and reading the Bedegat, or religious books. Of this last


kind of merit, there are three kinds: the first is the senseless
repetition of prayers and reading; the second, reading intelligently;
the last, is performing these exercises with strong desires and
feelings. Prayers are not addressed to any God, as there is none
existing now for this world. Gaudama, at his death, advised that, in
addition to obeying his laws, his relics and image should be
worshiped, and temples be built to his honor till the next Boodh
came.

Votive offerings of fruit, rice and flowers are made to priests or


placed in temples. The prayers consist of the repetition of soliloquies
that express our liability to bodily evils and to mental suffering, and
our inability to escape. Also of protestations of this kind, [pg 197] “I
will not lie;” “I will not steal;” “I will not kill,” etc.
There are four Sabbaths or days for public worship each month,
when the people go with votive offerings and prayers to the temple
of Gaudama, but they have no general united worship.

The Boodhists have a hierarchy very much like the Catholic church,
with varied grades and ranks. The priests are required to practice
celibacy, and are mainly supported by voluntary gifts from the
people.

They reside in buildings erected especially for them, and as celibacy


and the avoidance of women are enjoined on all, these
establishments very much resemble Catholic monasteries. Few of the
priests preach, and only by special request, after which, presents are
made to them. They attend funerals only when invited, and then
expect presents. Part of them spend some time in teaching novitiate
priests, but most of them, regarding work as unprofessional, spend
their time in sheer idleness. It is the rule that each priest
perambulate the streets every morning till he receives boiled rice
enough for his daily wants. The higher class of priests avoid this. In
Burmah the priests are at the rate of one to every thirty persons,
and they are well supported by the people, and without interference
from the government to enforce it.

As to the motives that sustain this religion, there being no God to


the Boodhist, all motives arising from relations and regard to him are
excluded. All the motives presented appeal to hope of good and fear
of evil to self. Those who attain a certain measure of merit in
obeying Gaudama's teachings go to some of the celestial regions,
according to their attainments. [pg 198] These consist of twenty-six
heavens, one above another, which offer various degrees of
enjoyment according to merit obtained.

There are eight principal hells; four that torment with cold and four
with heat. In the other hells are other sufferings, although not
connected with heat and cold. Worms bite, bowels are torn out,
limbs are racked, bodies are lacerated, they are pierced with hot
spits, crucified head downward, gnawed by dogs, torn by vultures.
These are described with minuteness in the Bedegat and often
depicted by the native artists in drawings, reminding one of Dante's
Inferno illustrated.

For killing a parent or a priest a man will suffer in one of the hells of
fire for inconceivable millions of ages. Denying the doctrines of
Gaudama incurs eternal suffering in fire. Insulting women, old men
or priests, receiving bribes, selling intoxicating drinks and parricide,
are punished in the worst hell.

Merit gained by any good conduct in these hells enables the person
to rise even to the celestial regions.

The souls of all the universe have existed from eternity,


transmigrating for ever, and thus rising and falling in the scale of
existence according to the degrees of merit at each birth. This is
decided not by any deity but by immutable fate. In passing through
these changes the amount of sorrow is incalculable. The Bedegat
declares that the tears shed by one soul in its various changes are
so great that the ocean in comparison is but a drop. Sorrow is
declared to be the inevitable attendant of all existence, and
therefore “the chief end,” and the highest reward of Boodhism is,
annihilation.

[pg 199]
The system of Boodhism commenced about six hundred years
before Christ, and has pervaded eastern, central and southern Asia
about as long and as fully as Christianity has pervaded Europe. The
Burman empire, where this account of that faith was obtained,
presents the most favorable results of this system on the character
and condition of its votaries.

In China, Buddhism (another name for Boodhism) is the popular


religion. With it is associated Confucianism, which is a system of
morals and politics instituted by Confucius, B. C. 550, which teaches
nothing in regard to any God or a future state. With them co-exist
the sect of Laotze, which is a kind of rationalism. Most of the
temples and priests are those of Boodh or Budda, but there is no
such organized priesthood as in Burmah, nor is this religion
maintained by governmental power. It is also considerably modified
by the more ancient system of polytheism.

In Thibet and Tartary, the religion of the Grand Lama chiefly prevails,
which is one form of Boodhism.

In western India, Brahmanism is in constant warfare with Boodhism,


and the two systems are perfectly antagonistic. Brahmanism teaches
one eternal deity and three hundred and thirty-three millions of
other gods, with hosts of idols representing them; Boodhism has no
deity at all, and only one image, that of Gaudama. Brahmanism
enjoins sacrifices; Boodhism forbids killing. Brahmanism requires
atrocious tortures; Boodhism inculcates fewer austerities than even
Popery. Brahmanism makes lying, fornication and theft sometimes
commendable, and describes the gods as excelling in such crimes;
Boodhism never confounds [pg 200] right and wrong, and never
excuses any sin. Brahmanism makes the highest good or chief end
of man to be absorption into the supreme deity; Boodhism makes
annihilation the highest hope and aim of existence. These two
systems, together with Mohammedanism, so prevail in Hindostan
that the distinct results of each can never be compared. These are
the prevailing religions in the most advanced pagan nations at the
present time; and of the two, Boodhism is the best, and probably
has been the most fairly tested in Burmah.

In past ages the two most highly developed heathen nations were
those of Greece and Rome, and of their religion we have the fullest
records. It is not probable that any one will consider their system of
religion superior to this now exhibited of modern paganism.

The result is that the most highly developed heathen nations, as yet,
have attained but very imperfectly the system of common sense.
No heathen religion ever taught an eternally-existing Creator, perfect
in knowledge, wisdom, power and benevolence. None ever taught
that the chief end of our Creator is happiness-making on the
greatest possible scale. None ever taught that this also is the chief
end for which man is created. None ever taught that right moral
action, or true virtue, consists in good willing toward the Creator,
toward self, and toward our fellow-beings, according to the laws of
the Creator, so that every mind shall make the good of self
subordinate to the general good. None ever taught that all questions
of right and wrong, or what is for the best, are to be decided with
reference to the risks and dangers of a future life. None ever
presented communion with, and the care, sympathy, sacrifices, and
example of a “long-suffering” [pg 201] Creator, as motives to secure
virtuous self-sacrifice from his creatures. If all this is taught by
revelations from God in the Bible, it is what was never taught by any
other religion yet known on earth.

In the history of the heathen world, we find anxious inquiries on


these subjects pressing on every thoughtful spirit. Who made this
world with its profound and ceaseless sorrows? Are there contending
deities, and are the malignant powers in the ascendant? If there be
one supreme Creator of all, is he propitious or hostile to a race so
guilty as ours? Does he feel any pity or sympathy for our profound
ignorance, our infinite sorrows? Can we do any thing to gain his help
in our darkness and misery? Where do we go when we die? Does
our short and painful span of being end in eternal night, or are we to
go on in another career of similar suffering and change? When we
lay our beloved ones in the grave, shall we ever meet them again, or
is “the only proper utterance of a broken heart, vale, vale, in
eternum vale?”

These have been the mournful questionings of every age and every
race, while the wisest sages of the wisest nations, without a
revelation, have been unable to give any satisfactory reply.
Greece and Rome were the most civilized of all ancient nations, and
they give us Socrates, Plato, Aristotle and Cicero, as their best and
wisest men, who most deeply pondered these great questions.
Aristotle held to one superior deity, but taught that the stars are true
and eternal deities. Cicero leads to the belief of many gods, and
approves of worshiping distinguished men as gods. Socrates held to
a plurality [pg 202] of deities, and also to transmigration. He held
that the common sort of good men will go into the forms of bees,
ants, and other animals of a mild and social kind. Plato held to two
principles, God and matter, and that God was not concerned either in
the creation or government of this world. He argued for the
immortality of the soul on the ground of its pre-existence, and
concludes some of his speculations thus:

“We can not of ourselves know what will be pleasing to God, or what
worship to pay him; but it is needful that a lawgiver be sent from
heaven. Such an one do I expect, and O how greatly do I desire to
see him, and who he is!”
Chapter XXXI. Augustinian Creeds and
Theologians Teach the Common-Sense
System.

In the former portion of this work the Augustinian theory, with the
system based on it, has been presented as it is taught by creeds and
theologians. In contrast with it, has been presented the common-
sense system of religion as evolved by reason and experience.

The evidence will now be presented, to show that those who teach
the Augustinian system, at the same time teach the main points of
the common-sense system; and where the two systems are
contradictory, that they teach both sides of the contradiction, at
once affirming and denying the same things.

[pg 203]
A leading feature of the common-sense system is, that the nature of
the human mind is our only guide to the natural attributes of God.

It will now be shown that leading theologians and metaphysicians of


the Augustinian school teach the same.

The Calvinistic theologians of New England have been universally


acknowledged as among the most acute and profound
metaphysicians in the world. At the head of these stands President
Jonathan Edwards. In reference to our modes of gaining a
knowledge of God, he says:
“If respect to the Divine Being is of any importance, then
speculative points are of importance, for the only way we can
know what he is, is by speculation.”

Dr. Woods, for near half a century a leading theological teacher of


New England, says:

“All our particular conceptions of God may be found to take


their rise from the conceptions we form of created
intelligences.”

Dr. Emmons, a distinguished New England divine, says of man:

“In the very frame and constitution of his nature he still bears
the natural image of his Maker. In a word, man is the living
image of the living God, in whom is displayed more of the
divine nature and glory than in all the works and creatures of
God upon earth.”

Dr. Taylor, the New Haven divine, says:

“The only ultimate source of knowledge, and ultimate umpire


of truth, is the knowing mind.”

[pg 204]
The celebrated Scotch metaphysician, Sir W. Hamilton, says:

“We can know God only as we know ourselves.”

In proof of this from the Bible, these writers quote from the Apostle
James, that “men are made after the similitude of God.”
Another leading feature of the common-sense system is the position,
that we can discover the chief end or design of the Creator, by the
nature of his works, and that this end is to produce the greatest
possible happiness with the least possible evil.

It will now be shown that leading theologians teach the same.

President Edwards, in his Dissertation concerning the end for which


God created the world, teaches that

“What God had respect to as an ultimate end of his creating


the world, was to communicate of his own infinite fullness of
good.”

He teaches that God is in no way dependent on his creatures for


happiness, but that his enjoyment consists in outpouring his own
good to his vast family.

No one can read that essay without perceiving that, though


disconnected passages may make a different impression, the above
is a correct statement of the doctrine of that dissertation.

It is supposed that this view has been assented to by most of those


American and European theologians who most strenuously defend
the Augustinian system.

The end or design of mind being ascertained, its right mode of


action is thus determined. Accordingly we shall find that the great
New England divines and metaphysicians, though they use different
language, [pg 205] all express the same idea in defining true virtue
or holiness.

Thus President Edwards taught, as his son states, that

You might also like