Usage-Driven Database Design: From Logical Data Modeling through Physical Schema Definition 1st Edition George Tillmann (Auth.) instant download
Usage-Driven Database Design: From Logical Data Modeling through Physical Schema Definition 1st Edition George Tillmann (Auth.) instant download
https://textbookfull.com/product/usage-driven-database-design-
from-logical-data-modeling-through-physical-schema-
definition-1st-edition-george-tillmann-auth/
https://textbookfull.com/product/logic-in-high-definition-trends-
in-logical-semantics-alessandro-giordani/
https://textbookfull.com/product/data-visualisation-a-handbook-
for-data-driven-design-andy-kirk/
https://textbookfull.com/product/data-visualisation-a-handbook-
for-data-driven-design-andy-kirk-2/
https://textbookfull.com/product/cracking-the-personality-code-
predicting-personality-from-smartphone-sensor-data-app-usage-
data-1st-edition-swaviman-kumar/
Analyzing Data through Probabilistic Modeling in
Statistics 1st Edition Dariusz Jakóbczak
https://textbookfull.com/product/analyzing-data-through-
probabilistic-modeling-in-statistics-1st-edition-dariusz-
jakobczak/
https://textbookfull.com/product/dynamic-mode-decomposition-data-
driven-modeling-of-complex-systems-1st-edition-j-nathan-kutz/
https://textbookfull.com/product/building-a-columnar-database-on-
ramcloud-database-design-for-the-low-latency-enabled-data-
center-1st-edition-christian-tinnefeld/
https://textbookfull.com/product/your-life-in-numbers-modeling-
society-through-data-pablo-jensen/
George Tillmann
Usage-Driven Database Design: From Logical Data Modeling through Physical
Schema Definition
George Tillmann
Ship Bottom, New Jersey, USA
ISBN-13 (pbk): 978-1-4842-2721-3 ISBN-13 (electronic): 978-1-4842-2722-0
DOI 10.1007/978-1-4842-2722-0
Library of Congress Control Number: 2017938116
Copyright © 2017 by George Tillmann
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical
way, and transmission or information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark
symbol with every occurrence of a trademarked name, logo, or image we use the names, logos,
and images only in an editorial fashion and to the benefit of the trademark owner, with no
intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
they are not identified as such, is not to be taken as an expression of opinion as to whether or not
they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the
date of publication, neither the authors nor the editors nor the publisher can accept any legal
responsibility for any errors or omissions that may be made. The publisher makes no warranty,
express or implied, with respect to the material contained herein.
Managing Director: Welmoed Spahr
Editorial Director: Todd Green
Acquisitions Editor: Jonathan Gennick
Development Editor: Laura Berendson
Coordinating Editor: Jill Balzano
Copy Editor: Kim Wimpsett
Compositor: SPi Global
Indexer: SPi Global
Artist: SPi Global
Cover image designed by Freepik
Distributed to the book trade worldwide by Springer Science+Business Media New York,
233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505,
e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is
a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc
(SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail rights@apress.com, or visit www.apress.com/
rights-permissions.
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook
versions and licenses are also available for most titles. For more information, reference our Print
and eBook Bulk Sales web page at www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is available
to readers on GitHub via the book’s product page, located at www.apress.com/9781484227213.
For more detailed information, please visit www.apress.com/source-code.
Printed on acid-free paper
For
Eva Marie
and in memory of
Catherine.
Contents at a Glance
■
■Part I: Introduction��������������������������������������������������������� 1
■
■Chapter 1: Introduction to Usage-Driven Database Design������������ 3
■
■Part II: Logical Data Modeling�������������������������������������� 13
■
■Chapter 2: The E-R Approach������������������������������������������������������� 15
■
■Chapter 3: More About the E-R Approach������������������������������������� 23
■
■Chapter 4: Building the Logical Data Model��������������������������������� 45
■
■Chapter 5: LDM Best Practices����������������������������������������������������� 69
■
■Chapter 6: LDM Pitfalls����������������������������������������������������������������� 99
■
■Chapter 7: LDM Perils to Watch For�������������������������������������������� 119
■
■Part III: Physical Schema Definition��������������������������� 133
■
■Chapter 8: Introduction to Physical Database Design���������������� 135
■
■Chapter 9: Introduction to Physical Schema Definition�������������� 165
■■Chapter 10: Transformation: Creating the Physical
Data Model��������������������������������������������������������������������������������� 185
■
■Chapter 11: Utilization: Merging Data and Process�������������������� 211
■
■Chapter 12: Formalization: Creating a Schema�������������������������� 233
■
■Chapter 13: Customization: Enhancing Performance����������������� 257
v
■ Contents at a Glance
■
■Chapter 14: The Data Warehouse����������������������������������������������� 287
■
■Chapter 15: The Big Data Decision Support System������������������� 301
■
■Part IV: Where from Here?������������������������������������������ 315
■
■Chapter 16: A Look Ahead���������������������������������������������������������� 317
■
■Part V: Appendixes����������������������������������������������������� 327
■
■Appendix A: Glossary����������������������������������������������������������������� 329
■
■Appendix B: Logical Data Modeling Definitions�������������������������� 349
■
■Appendix C: Physical Schema Definition Object Definitions������� 353
■
■Appendix D: Formulas Used in This Book����������������������������������� 359
■
■Appendix E: List of U3D Deliverables������������������������������������������ 361
Index���������������������������������������������������������������������������������������������� 365
vi
Contents
■
■Part I: Introduction��������������������������������������������������������� 1
■
■Chapter 1: Introduction to Usage-Driven Database Design������������ 3
Database Design Principle 1: Separation Principle��������������������������������� 5
Database Design Principle 2: Distinction Principle���������������������������������� 6
The Difference Between Separation and Distinction������������������������������������������������ 8
Notes����������������������������������������������������������������������������������������������������� 12
■
■Part II: Logical Data Modeling�������������������������������������� 13
■
■Chapter 2: The E-R Approach������������������������������������������������������� 15
A Little Data Modeling History��������������������������������������������������������������� 17
Some Important Definitions������������������������������������������������������������������� 18
Logical Data Modeling Objects�������������������������������������������������������������� 19
Entities�������������������������������������������������������������������������������������������������������������������� 19
Type-Instance Distinction��������������������������������������������������������������������������������������� 20
vii
■ Contents
Relationships���������������������������������������������������������������������������������������������������������� 20
Attributes���������������������������������������������������������������������������������������������������������������� 21
Notes����������������������������������������������������������������������������������������������������� 22
■
■Chapter 3: More About the E-R Approach������������������������������������� 23
More About Relationships��������������������������������������������������������������������� 23
Membership Class�������������������������������������������������������������������������������������������������� 23
Degree�������������������������������������������������������������������������������������������������������������������� 27
Relationship Constraints����������������������������������������������������������������������������������������� 29
■
■Chapter 4: Building the Logical Data Model��������������������������������� 45
The Interview Process��������������������������������������������������������������������������� 46
Gather Information and Review������������������������������������������������������������������������������ 47
Analyze Information������������������������������������������������������������������������������������������������ 48
Construct Model����������������������������������������������������������������������������������������������������� 49
Repeat as Necessary���������������������������������������������������������������������������������������������� 49
viii
■ Contents
Deliverables������������������������������������������������������������������������������������������� 63
Examples of Deliverables���������������������������������������������������������������������� 64
Sample Data Dictionary, Data Object Definitions���������������������������������������������������� 65
Notes����������������������������������������������������������������������������������������������������� 67
■
■Chapter 5: LDM Best Practices����������������������������������������������������� 69
Abbreviations���������������������������������������������������������������������������������������� 70
Almost Unique Identifiers���������������������������������������������������������������������� 71
Clarity���������������������������������������������������������������������������������������������������� 72
Compound Unique Identifiers���������������������������������������������������������������� 72
Conceptual Integrity������������������������������������������������������������������������������ 73
Conjunctive Relationships��������������������������������������������������������������������� 75
ix
■ Contents
Exclusive Relationships������������������������������������������������������������������������� 78
Group Attributes������������������������������������������������������������������������������������ 79
Level of Abstraction������������������������������������������������������������������������������� 79
Many-to-Many Relationships���������������������������������������������������������������� 80
N-ary Relationships������������������������������������������������������������������������������� 82
N-ary Relationships and Membership Class����������������������������������������������������������� 83
Naming Objects������������������������������������������������������������������������������������� 86
Multiple Names������������������������������������������������������������������������������������������������������� 86
Naming Conventions����������������������������������������������������������������������������������������������� 87
Name Uniqueness��������������������������������������������������������������������������������������������������� 87
Naming Convention Goals��������������������������������������������������������������������������������������� 88
Null Attributes���������������������������������������������������������������������������������������� 91
There Be Blanks in Them Thar Nulls����������������������������������������������������������������������� 91
x
■ Contents
xi
■ Contents
■
■Chapter 7: LDM Perils to Watch For�������������������������������������������� 119
Associatives Related to Other Associatives����������������������������������������� 119
Diagrammable Objects������������������������������������������������������������������������ 120
Disassociated Entity Clusters (“Islands”)�������������������������������������������� 121
Duplicate Unique Identifiers���������������������������������������������������������������� 122
One Entity, Two or More Identifiers����������������������������������������������������������������������� 122
One Identifier, Two or More Entities���������������������������������������������������������������������� 122
Spiderwebs����������������������������������������������������������������������������������������� 128
Too Many Blanks or Nulls�������������������������������������������������������������������� 129
Too Many Recursives�������������������������������������������������������������������������� 130
Next U3D Phase: Physical Schema Definition������������������������������������� 131
Notes��������������������������������������������������������������������������������������������������� 131
■
■Part III: Physical Schema Definition��������������������������� 133
■
■Chapter 8: Introduction to Physical Database Design���������������� 135
A Short Incondite History of Automated Information Management
(or, a Sequential Look at Random Access)������������������������������������������ 136
Information Management Era 1: Sequential Processing�������������������������������������� 136
Information Management Era 2: The First Random Access DBMS����������������������� 137
Information Management Era 3: Inverted File Systems���������������������������������������� 150
xii
■ Contents
Summary��������������������������������������������������������������������������������������������� 183
■■Chapter 10: Transformation: Creating the Physical
Data Model��������������������������������������������������������������������������������� 185
Task 1.1: Translation���������������������������������������������������������������������������� 186
Activity 1.1.1: Transform LDM Objects to PDM Objects���������������������������������������� 186
Activity 1.1.2: Diagram the Objects���������������������������������������������������������������������� 194
xiii
■ Contents
xiv
■ Contents
■
■Chapter 12: Formalization: Creating a Schema�������������������������� 233
Task 3.1: Environment Designation����������������������������������������������������� 234
Hierarchical Systems�������������������������������������������������������������������������������������������� 237
Network Systems������������������������������������������������������������������������������������������������� 237
Relational Systems����������������������������������������������������������������������������������������������� 237
Object-Oriented���������������������������������������������������������������������������������������������������� 238
NoSQL������������������������������������������������������������������������������������������������������������������� 238
DBMS Product and Version Selection������������������������������������������������������������������� 238
Task 3.2: Constraint Compliance��������������������������������������������������������� 239
Pseudocode…Again��������������������������������������������������������������������������������������������� 239
Activity 3.2.1: Map Rationalized Physical Data Model to the
Data Architecture�������������������������������������������������������������������������������������������������� 241
Data Items������������������������������������������������������������������������������������������������������������ 248
Activity 3.2.2: Create a DBMS Product/Version-Specific Functional Physical
Database Design��������������������������������������������������������������������������������������������������� 251
xv
■ Contents
■
■Chapter 14: The Data Warehouse����������������������������������������������� 287
The Data Warehouse��������������������������������������������������������������������������� 288
Data Warehouse Architecture������������������������������������������������������������������������������� 290
■
■Part IV: Where from Here?������������������������������������������ 315
■
■Chapter 16: A Look Ahead���������������������������������������������������������� 317
We Need to Ask the Awkward Questions��������������������������������������������� 318
Tools Need to Take Usage into Account����������������������������������������������� 320
The One and Only DBMS��������������������������������������������������������������������� 320
Better Training������������������������������������������������������������������������������������� 324
Notes��������������������������������������������������������������������������������������������������� 326
xvi
■ Contents
■
■Part V: Appendixes����������������������������������������������������� 327
■
■Appendix A: Glossary����������������������������������������������������������������� 329
■
■Appendix B: Logical Data Modeling Definitions�������������������������� 349
Entity��������������������������������������������������������������������������������������������������� 349
Relationship���������������������������������������������������������������������������������������� 350
Attribute���������������������������������������������������������������������������������������������� 350
Domain������������������������������������������������������������������������������������������������ 351
■
■Appendix C: Physical Schema Definition Object Definitions������� 353
Record������������������������������������������������������������������������������������������������� 353
Linkage������������������������������������������������������������������������������������������������ 354
Data Item��������������������������������������������������������������������������������������������� 355
Domain������������������������������������������������������������������������������������������������ 355
Cluster������������������������������������������������������������������������������������������������� 356
Partition����������������������������������������������������������������������������������������������� 356
Index���������������������������������������������������������������������������������������������������� 357
■
■Appendix D: Formulas Used in This Book����������������������������������� 359
■
■Appendix E: List of U3D Deliverables������������������������������������������ 361
Index���������������������������������������������������������������������������������������������� 365
xvii
About the Author
xix
Preface
Every year Stack Overflow (a programmer Q&A site on the Stack Exchange Network)
surveys readers about a number of issues. A 2015 survey of more than 26,000 system
developers found that only 38 percent had a computer science degree and 33 percent had
never taken even one college computer science course. An amazing 42 percent said that
they were totally self-taught.1
These miserable statistics are for system developers as a whole. Although there
are no numbers readily available, those who have spent decades in the database arena
indicate that database designer training is even worse. Universities and companies
spend more time and more education dollars on the process side of system development,
leaving the data side the underserved orphan.
What Happened?
Many database designs are terrible because many database designers are undertrained.
Don’t blame the designers—they are working in a system in which all the cards are
stacked against them. The problem is a poor system of educating database staff.
Anecdotal information indicates that the average database designer or database
programmer has about 10 percent of the training in information management that the
average process-oriented system designer or programmer has in process management.
The majority of database designers learned their trade from database management
system (DBMS) vendor courses or from reading books and trade publications.
The training goals of the average DBMS vendor, understandably, do not focus on
providing a balanced data management education but rather on teaching customer staff
how to use its products. The student might come away schooled in the use of a particular
DBMS, but any understanding of the fundamentals of what it is to be a database
management system are serendipitous.
xxi
■ Preface
Most database books are no better. There are many titles on information
management. Some purport to focus on logical data modeling, others on database
design, and the majority on both. Some mention a specific data architecture (hierarchical,
relational, object-oriented, NoSQL, etc.) or product (Oracle, DB2, Cassandra, etc.), while
the covers of most of these books make no claim to an alignment or preference for a
specific architecture or product. However, the contents of these books tell a different
story. Almost all of the “generic” books are saturated with relational terms, relational
concepts, and relational thinking. Any concession to generic or nonrelational information
management is marginal. The reader is left with a tunnel-vision view of the field.
Worse, techniques, such as logical data modeling, which should have nothing to do
with any particular DBMS architecture or any physical design issues, are jam-packed with
DBMS-specific—usually relational—terminology and thinking. For example, logical data
modeling relationships are represented using foreign keys, primary keys are arbitrarily
selected from the pool of candidate keys, and many-to-may relationships are “resolved”
with junction tables.
There is nothing wrong with the relational model. In fact, no database education
would be complete without a good understanding of the relational model and relational
database management systems. However, relational is not the only architecture, not the
only DBMS, and not the only way to design a database. A myopic education might train
the reader in how to use a specific DBMS but imparts little of what it is to be a DBMS.
Learning about the relational model is essential; learning about it exclusively is harmful.
Another common characteristic of most database design books is that they are
written by academics. Many of these books are excellent, and every database professional
should have a bookshelf bristling with their titles. But a balanced education needs more
than just a classroom view of the information management world. Formal instruction is
good, but it just doesn’t go far enough. Most database designers will not find themselves
in the rarefied air of the ivory tower but rather deep down in the corporate trenches with,
as Johnny Cash put it, “the mud and the blood and the beer.” They need the practical as
well as the academic, the team room as well as the classroom.
This book’s approach to data management is far less theoretical and far less
dogmatic than the books sitting on that bookshelf. Rather, it focuses on what works,
what doesn’t work, and what to avoid at all costs. It includes some of the knowledge,
techniques, and tricks that can turn a disaster into a success. The major influences on this
book certainly include numerous academic authors but also incorporate the experiences
of database developers, designers, and users all over the world who did it right or,
unfortunately, did it wrong.
xxii
■ Preface
Process models have their own techniques geared to documenting business functions.
What is needed is a way to merge process and data so that the database design represents
the union of definition and use. This is the database design missing link—the critical
component that marries these two distinct elements. When this link is missing, the
database design focuses almost entirely on the definition of data without taking use
into account. These databases tend to be poor performers and the cause of numerous
end-user complaints. Alternatively, a database design can be based solely on use. These
databases become prematurely old, requiring costly and constant maintenance and
updates.
Usage-Driven Database Design (U3D) gives the database designer the necessary
tools to resolve the missing link problem. Using U3D, database designs can have the
resilience of the data model with the functional responsiveness of the process model.
U3D can eliminate the database design missing link.
The goal of this book, therefore, is not so much education as inheritance—to pass
on to the database designer, database administrator (DBA), or database programmer
the technique and tricks uncovered and used by some of the best and some of the worst
database people in the world.
A Solution
This book is divided into five parts. Part I consists of a single chapter; Chapter 1,
“Introduction to Usage-Driven Database Design,” introduces the four database design
principles. Although these principles are geared toward database design, they are, in fact,
a sound starting point for any system development activity. The chapter ends with the
introduction of usage-driven database design, an end-to-end framework for developing
a functioning database, starting with the logical data model and ending with a physical
database schema.
Part II focuses on logical data modeling. Chapter 2, “The E-R Approach,” introduces
Peter Chen’s entity-relationship (E-R) approach, while Chapter 3, “More About the E-R
Approach,” focuses on more advanced logical data modeling topics. Chapter 4, “Building
the Logical Data Model,” uses the Usage-Driven Database Design: Logical Data Modeling
phase as a template to tackle the real-world tasks of actually building a logical data model
for an enterprise.
Chapter 5, “LDM Best Practices,” presents lessons learned from the database
trenches. Chapter 6, “LDM Pitfalls,” gives advice on what to avoid when data modeling.
Chapter 7, “LDM Perils to Watch For,” presents some logical data modeling cautionary
tales.
In Part III, the logical data model becomes a functioning database schema.
Chapter 8, “Introduction to Physical Database Design,” presents a limited history of data
management; however, the focus is gaining practical rather than historical insight. The
concepts presented are used in later chapters for creating great databases.
Chapter 9, “Introduction to Physical Schema Definition,” introduces the four steps in
the Usage-Driven Database Design: Physical Schema Definition phase that will turn the
logical data model into a physical database schema.
Chapter 10, “Transformation: Creating the Physical Data Model,” converts the logical
data model into a physical data model.
xxiii
■ Preface
Chapter 11, “Utilization: Merging Data and Process,” modifies the physical data
model to reflect exactly how an application will use the database. This is an important
chapter because many database design approaches do not adequately take data usage
into account.
Chapter 12, “Formalization: Creating a Schema,” converts the modified physical data
model into a functioning physical database schema and subschemas.
Chapter 13, “Customization: Enhancing Performance,” addresses those situations
where a simple database design cannot handle the load that will be placed on it. This
step introduces performance-enhancing techniques (software, hardware, NoSQL, etc.)
that can be applied to almost any situation to accommodate almost any performance
requirements.
Chapter 14, “The Data Warehouse,” shows how U3D can be used to construct a data
warehouse to support a decision support system.
Chapter 15, “The Big Data Decision Support System,” shows how U3D can be used
with nontraditional data management products, such as Hadoop, to accommodate
unstructured Big Data.
Part IV contains a single chapter, Chapter 16, “A Look Ahead,” which discusses where
the DBMS community (teachers, vendors, and technical users) are or should be going.
Part V contains five appendixes that include a glossary, data management object
definitions, formulas, and a list of U3D deliverables.
This book is aggressively practical and generic. For example, it vigorously keeps
logical data modeling logical, while holding off on physical issues until physical database
design—not to justify some philosophical or theoretical construct but for the practical
reason that it greatly increases the chances of developing a successful database design.
It is DBMS generic or agnostic in that it does not tie the hands of the developer who
is attempting to solve real-world information management problems. The right solution
might involve a relational DBMS or it might require a NoSQL DBMS. Or, more likely, the
DBMS choice was made some time ago, and now the database designer needs help in
making the best of an imperfect DBMS situation.
In summary, this book is for the undervalued data management professional who
has to transform a combination of glossy DBMS vendor brochures and dry textbook
commentary into a functioning fundamental part of the enterprise.
George Tillmann
george_tillmann@gmx.com
georgetillmann@optonline.net
Note
1.
The Stack Exchange, http://stackoverflow.com/research/
developer-survey-2015#profile-education
xxiv
PART I
Introduction
CHAPTER 1
Introduction to Usage-Driven
Database Design
In 2015, the IT industry cost the world about $3.5 trillion according to Gartner, Inc., a
research and advisory firm.1 The Standish Group2 reported that, in the United States
alone, about $250 billion of that was spent on application development. Less than 30
percent (about $72 billion) was spent on successful undertakings, with almost $50
billion written off on failed projects. In the United States, another $130 billion was
spent on projects that were completed but were over budget, late, or lacked promised
functionality. A pretty dismal picture. Why?
In almost any area of endeavor, there are experts and there are neophytes. You can
usually recognize the neophyte by his nose in the “how to” book. The expert? He is just
standing there, comfortable that any needed knowledge is in his head. Let’s call it the
Confident Expert Syndrome.
However, in at least three professions there is no Confident Expert Syndrome. The
first is airline pilot. Any senior airline captain will tell you that they meticulously go
through both the takeoff and landing checklists. They will also tell you that they are wary
of flying with a copilot who cuts checklist corners. That’s the reason they have lived long
enough to become a senior captain.
The second profession where the Confident Expert Syndrome does not apply is
project management. The seasoned project manager is the one poring over the system
development manuals and plumbing the depths of the project plan. They know that
success flows from following the system development life cycle (SDLC) methodology
and the project plan to the letter. The neophyte project manager is the one who feels that
steps can be left out or shortened or that time can be made up in future tasks. Successful
project managers know that if you want to bring the project in on time, on budget, and
fully functional, then you must complete every step.
The third profession where the Confident Expert Syndrome does not (or should not)
exist is database designer. Experienced database designers are the ones who perform all
the necessary design functions, in their proper sequence, leaving out nothing. It is the
rookie designers who think that the single database design course they took, taught by
their database management system (DBMS) vendor, is all they need to design quality
databases.
However, there is one significant difference between poor project management or
poor analysis or poor programming and poor database design. Poor project management
results in failed projects, and poor analysis and poor programming result in programs
that will not compile or run. Poor database design, on the other hand, results (far too
often) in a database that seems to work just fine.
Database management systems, by their very nature, cover up a multitude of
design errors. It might run slow—it might run very slow—but it will usually work. An IT
shop might live with a poor database design for years, blaming the DBMS software, the
hardware, the system software, or even the application programmers for performance
that is the result of very poor up-front database design decisions.
How do you design good databases? Do what the pros do—follow the tried and true
steps for creating a great design. However, while there are a few good database design
techniques and guidelines, many if not most of them have not been incorporated into a
full, end-to-end database design method. Application developers have great end-to-end
methods; in fact, many system development life-cycle products, if followed, can lead
to great applications. Project managers have a host of available project management
methods, techniques, and tools to choose from. Database designers? Not so much…at
least until now…sort of.
Doesn’t such a process already exist? Well, yes and no. Snippets of a method are
available and in use, but there is no end-to-end solution. However, that is not the only
problem.
Looking to the best practices of the experienced is not always enough. Even premier
development approaches, first-class training courses, and books by experts have a
common failing. They all lack an effective approach to merge the definition of the data
with the data’s use.
There are, or should be, two major inputs to any database design approach—the
logical data model and the logical process model (both are discussed in detail in later
chapters). Logical data modeling uncovers the definition of data, their characteristics,
and their relationship with other data. Logical data models are a static though longer-
term picture of an organization’s data. Logical process modeling documents how the data
move through the enterprise. Logical process models paint a shorter-lived albeit dynamic
picture of the information—the processes and procedures—an enterprise uses to go
about its business.
Many traditional database design approaches focus exclusively on the logical data
model, ignoring how the data will be used, resulting in poor performing systems. Other
database design approaches focus on the use of the data, while giving short shrift to
4
Chapter 1 ■ Introduction to Usage-Driven Database Design
the definition of data, resulting in databases that are expensive to maintain. The absent
component to effectively merge the dynamic process models with the static data models
is the database design missing link.
This book solves the missing link problem, providing a technique that effectively and
efficiently marries data and process. The result is a database design encapsulating the
stability and longevity of the logical data model with the functionality and applicability of
the logical process models.
The goal of this book is to provide the designer with the best thinking and best
practices on database design, gleaned from decades of hands-on experience working
with database designers in dozens of IT organizations. Where best practices are lacking or
acceptable methods or approaches do not exist, this book provides them.
The approach presented here is a composite of how to use what has worked and how
to avoid what hasn’t worked at all costs. But is it a database design methodology?
It is not really a method because a method should bristle with excruciatingly detailed
steps. This approach is better called a framework for developing databases—more than
an unrelated string of techniques but less than ten binders of forms to fill out.
The pillars of this approach are four database design principles.
5
Chapter 1 ■ Introduction to Usage-Driven Database Design
There are project management books that describe the disasters that await the
system developer who ignores this best practice. Unfortunately, database design books
often ignore this principle with similar disastrous consequences.
The hard reality is that data require the same insight and attention as processes do,
which is why the first principle of this book is to always, even with iterative development,
separate the what (logical design) from the how (physical design).
There is a corollary to Principle 1—call it the Real World Corollary. It states the
purpose of logical design is to document the real world, which, in this context, is the
business world. There are two parts to Corollary 1.
• Corollary 1 (a): A logical design is valid if, and only if, it reflects
the real (business) world.
• Corollary 1 (b): A logical design is invalid if it contains nonreal
(business) world objects or concepts. Invalid objects and
concepts include elements belonging in physical design such as
foreign keys, pointers, and disk drives.
6
Chapter 1 ■ Introduction to Usage-Driven Database Design
7
Chapter 1 ■ Introduction to Usage-Driven Database Design
Logical Design
Data Process
• Static • Dynamic
• Stable • Unstable
• Long life • Short life
• Based on definition • Based on use
This is not to say that the two teams do not need to communicate and coordinate
efforts. The deliverables of the two teams, while they need to reflect the work of the other,
also need to be independent of each other.
8
Exploring the Variety of Random
Documents with Different Content
"His courage was of a high order. He looked death in the face more
than a hundred times without trepidation. He braved the tortures
and the stake among the Iroquois, the treacherous stratagems of
the savages of the West, the rigorous winters of the Hudson Bay,
and the tropical heat of the Antilles.
"Of an adventurous nature, drawn irresistibly to regions unknown,
carried on by the enthusiasm of his voyages, always ready to push
out into new dangers, he could have been made by Fenimore
Cooper one of the heroes of his most exciting romances.
"The picture of his life consequently presents many contrasts. The
life of a brigand, which he led with a party of Iroquois, cannot be
explained away.
"He was blamable in a like manner for having deserted the flag of
France, his native country. The first time we might, perhaps, pardon
him, for he was the victim of grave injustice on the part of the
government of the colony.
"No excuse could justify his second desertion. He had none to offer,
not one. He avowed very candidly that he sought the service of
England because he preferred it to that of France.
"In marrying the daughter of Mr. John Kirke, he seems to have
espoused also the nationality of her family. As for him, he would
have needed to change the proverb, and, in the place of 'One who
marries a husband takes his country,' to say, 'One who marries a
wife takes her country.'
"The celebrated discover of the North-West, the illustrious Le
Verendrye, has as much as Radisson, and even more than he, of just
reason to complain of the ingratitude of France; yet how different
was his conduct!
"Just as his persecutions have placed upon the head of the first a
new halo of glory, so they have cast upon the brow of the second an
ineffaceable stain.
"Souls truly noble do not seek in treason the recompense for the
rights denied them."
(For a detailed chronological account of Radisson's life, see Appendix
B, page 487.)
CHAPTER VI.
FRENCH RIVALRY.
The golden lilies in danger—"To arrest Radisson"—The land called "Unknown"—A chain of
claim—Imaginary pretensions—Chevalier de Troyes—The brave Lemoynes—Hudson Bay
forts captured—A litigious governor—Laugh at treaties—The glory of France—Enormous
claims—Consequential damages.
LE MOYNE D'IBERVILLE.
A short time after this the Company showed its disapproval of
Governor Sargeant's course in surrendering Fort Albany so readily.
Thinking they could mark their disapprobation more strongly, they
brought an action against Governor Sargeant in the courts to recover
20,000l. After the suit had gone some distance, they agreed to refer
the matter to arbitration, and the case was ended by the Company
having to pay to the Governor 350l. The affair, being a family
quarrel, caused some amusement to the public.
The only place of importance now remaining to the English on
Hudson Bay was Port Nelson, which was near the French Fort
Bourbon. D'Iberville, utilizing the vessel he had captured on the Bay,
went back to Quebec in the autumn of 1687 with the rich booty of
furs taken at the different points.
These events having taken place at a time when the two countries,
France and England, were nominally at peace, negotiations took
place between the two Powers.
Late in the year 1686 a treaty of neutrality was signed, and it was
hoped that peace would ensue on Hudson Bay. This does not seem
to have been the case, however, and both parties blame each other
for not observing the terms of the Act of Pacification. D'Iberville
defended Albany Fort from a British attack in 1689, departed in that
year for Quebec with a ship-load of furs, and returned to Hudson
Bay in the following year. During the war which grew out of the
Revolution, Albany Fort changed hands again to the English, and
was afterwards retaken by the French, after which a strong English
force (1692) repossessed themselves of it. For some time English
supremacy was maintained on the Bay, but the French merely waited
their time to attack Fort Bourbon, which they regarded as in a
special sense their own. In 1694 D'Iberville visited the Bay, besieged
and took Fort Bourbon, and reduced the place with his two frigates.
His brother De Chateauguay was killed during the siege.
In 1697 the Bay again fell into English hands, and D'Iber ville was
put in command of a squadron sent out for him from France, and
with this he sailed for Hudson Bay. The expedition brought unending
glory to France and the young commander. Though one of his
warships was crushed in the ice in the Hudson Straits and his
remaining vessels could nowhere be seen when he reached the open
waters of the Bay, yet he bravely sailed to Port Nelson, purposing to
invest it in his one ship, the Pelican. Arrived at his station, he
observed that he was shut in on the rear by three English men-of-
war. His condition was desperate; he had not his full complement of
men, and some of those on board were sick. His vessel had but fifty
guns; the English vessels carried among them 124. The English
vessels, the Hampshire, the Dering, and the Hudson's Bay, all
opened fire upon him. During a hot engagement, a well-aimed
broadside from the Pelican sank the Hampshire with all her sails
flying, and everything on board was lost; the Hudson's Bay
surrendered unconditionally, and the Dering succeeded in making
her escape. After this naval duel D'Iberville's missing vessels
appeared, and the commander, landing a sufficient number of men,
invested and took Port Nelson. The whole of the Hudson Bay
territory thus came into the possession of the French. The matter
has always, however, been looked at in the light of the brilliant
achievement of this scion of the Le Moynes.
Few careers have had the uninterrupted success of that of Pierre Le
Moyne D'Iberville, although this fortune reached its climax in the
exploit in Hudson Bay. Nine years afterwards the brilliant soldier died
of yellow fever at Havana, after he had done his best in a
colonization enterprise to the mouth of the Mississippi which was
none too successful. Though the treaty of Ryswick, negotiated in this
year of D'Iberville's triumphs, brought for the time the cessation of
hostilities, yet nearly fifteen years of rivalry, and for much of the
time active warfare, left their serious traces on Hudson's Bay
Company affairs. A perusal of the minutes of the Hudson's Bay
Company during this period gives occasional glimpses of the state of
war prevailing, although it must be admitted not so vivid a picture as
might have been expected. As was quite natural, the details of
attacks, defences, surrenders, and parleys come to us from French
sources rather than from the Company's books. That the French
accounts are correct is fully substantiated by the memorials
presented by the Company to the British Government, asking for
recompense for losses sustained.
In 1687 a petition was prepared by the Hudson's Bay Company, and
a copy of it is found in one of the letter-books of the Company. This
deals to some extent with the contention of the French king, which
had been lodged with the British Government, claiming priority of
ownership of the regions about Hudson Bay. The arguments
advanced are chiefly those to which we have already referred. The
claim for compensation made upon the British Government by the
Company is a revelation of how seriously the French rivalry had
interfered with the progress of the fur trade. After still more serious
conflict had taken place in the Bay, and the Company had come to
be apprehensive for its very existence, another petition was laid
before His Majesty William III., in 1694. This petition, which also
contained the main facts of the claim of 1687, is so important that
we give some of the details of it. It is proper to state, however, that
a part of the demand is made up of what has since been known as
"consequential damages," and that in consequence the matter
lingered on for at least two decades.
The damages claimed were:—
1682. Captain Gillam and cargo on £ s. d.
Prince Rupert. (Captain and a
number of men, cargo, and ship
all lost in hostilities.) Governor
Bridgar and men seized and
carried to Quebec
Moderate damages 25,000 0 0
September French with two ships built a
1684. small house and interrupted
Indian trade
Damages 10,000 0 0
1685. French took Perpetuana and
cargo to Quebec
Damages 5,000 0 0
For ship, master, and men
Damages 1,255 16 3
1686. French destroyed three of
Company's ships at bottom of
Bay, and also three ships' stores,
etc., and took 50,000 beaver
skins, and turned out to sea a
number of His Majesty's subjects 50,000 0 0
1682-86. Five years' losses about Forts
(10,000 beaver skins yearly) 20,000 0 0
1688. Company's ships Churchill and
Young seized by the French 10,000 0 0
1692. Company sent out expedition to
retake Forts, which cost them 20,000 0 0
1686-93. French possessed bottom of the
Bay for seven years. Loss,
10,000l. a year 70,000 0 0
Damages 20,000 0 0
Total damages claimed £211,255 16 3
CHAPTER VII.
RYSWICK AND UTRECHT.
The "Grand Monarque" humbled—Caught napping—The Company in peril—Glorious Utrecht
—Forts restored—Damages to be considered—Commission useless.
When peace had been restored by the Treaty of Utrecht, the shores
of the Bay, which had been in the hands of the French since the
Treaty of Ryswick, were given over to Great Britain, according to the
terms of the Treaty; they have remained British ever since. The
Company, freed from the fears of overland incursions by the French
from Canada, and from the fleets that had worked so much mischief
by sea, seems to have changed character in the personnel of the
stockholders and to have lost a good deal of the pristine spirit. The
charge is made that the stockholders had become very few, that the
stock was controlled by a majority, who, year after year, elected
themselves, and that considering the great privileges conferred by
the Charter, the Company was failing to develop the country and was
sleeping in inglorious ease on the shores of Hudson Bay. Certain it is
that Sir Bibye Lake was re-elected Governor year after year, from
1720 to 1740.
It would appear, however, to have been a spirit of jealousy which
animated those who made these discoveries as to the Company's
inaction. The return of peace had brought prosperity to the traders;
and dividends to the stockholders began to be a feature of company
life which they had not known for more than a quarter of a century.
As we shall see, the stock of the Company was greatly increased in
1720, and preparations were being made by the Committee for a
wide extension of their operations.
About this time a man of great personal energy appears on the
scene of English commercial life, who became a bitter opponent of
the Company, and possessed such influence with the English
Government that the Company was compelled to make a strenuous
defence. This was Arthur Dobbs, Esq., an Irishman of undoubted
ability and courage. He conducted his plan of campaign against the
Company along a most ingenious and dangerous line of attack.
He revived the memory among the British people of the early
voyages to discover a way to the riches of the East, and appealed to
the English imagination by picturing the interior of the North
American Continent, with its vast meadows, splendid cascades, rich
fur-bearing animals, and numberless races of Indians, picturesquely
dressed, as opening up a field, if they could be reached, of lucrative
trade to the London merchants. To further his purpose he pointed
out the sluggish character of the Hudson's Bay Company, and
clinched his arguments by quoting the paragraph in the Charter
which stated that the great privileges conferred by generous Charles
II. were bestowed in consideration of their object having been "The
Discovery of a New Passage into the South Sea." Dobbs appealed to
the sacrifices made and the glories achieved in earlier days in the
attempt to discover the North-West Passage. In scores of pages, the
indefatigable writer gives the accounts of the early voyages.
We have but to give a passage or two from another author to show
what a powerful weapon Dobbs wielded, and to see how he
succeeded in reviving a question which had slumbered well nigh a
hundred years, and which again became a living question in the
nineteenth century.
This writer says:—"It would lead us far beyond our limits were we to
chronicle all the reasons urged, and the attempts made to 'finde out
that short and easie passage by the North-west, which we have
hitherto so long desired.' Under the auspices of the 'Old Worthies'
really—though ostensibly countenanced by kings, queens, and
nobles—up rose a race of men, daring and enthusiastic, whose
names would add honour to any country, and embalm its history.
"Commencing with the reign of Henry VII., we have first, John Cabot
(1497), ever renowned; for he it was who first saw and claimed for
the 'Banner of England,' the American continent. Sebastian, his son,
follows in the next year—a name honourable and wise. Nor may we
omit Master Robert Thorne of Bristol (1527); Master Hore (1536);
and Master Michael Lok (1545), of London—men who knew
'cosmography' and the 'weighty and substantial reasons' for 'a
discovery even to the North Pole.' For a short time Arctic energy
changed its direction from the North-west to the North-east
(discoveries of the Muscovy Company), but wanting success in that
quarter, again reverted to the North-west. Then we find Martin
Frobisher, George Best, Sir Humphrey Gilbert, James Davis, George
Waymouth, John Knight, the cruelly treated Henry Hudson, James
Hall, Sir Thomas Button, Fotherbye, Baffin and Bylot, 'North-west'
Luke Fox, Thomas James, &c.
"Thus, in the course of sixty years—now breaking the icy fetters of
the North, now chained by them; now big with high hope 'of the
Passage,' then beaten back by the terrific obstacles, as it were,
guarding it—notwithstanding, these men never faltered, never
despaired of finally accomplishing it. Their names are worthy to be
held in remembrance; for, with all their faults, all their strange
fancies and prejudices, still they were a daring and glorious race,
calm amid the most appalling dangers; what they did was done
correctly, as far as their limited means went; each added something
that gave us more extended views and a better acquaintance with
the globe we inhabit—giving especially large contributions to
geography, with a more fixed resolution to discover the 'Passage.' By
them the whole of the eastern face of North America was made
known, and its disjointed lands in the North, even to 77 deg. or 78
deg. N. Their names will last while England is true to herself."
Mr. Dobbs awakened much interest among persons of rank in
England as to the desirability of finding a North-West Passage.
Especially to the Lords of the Admiralty, on whom he had a strong
hold, did he represent the glory and value of fitting out an
expedition to Hudson Bay on this quest.
Dobbs mentions in his book the unwilling efforts of the Hudson's Bay
Company to meet the demand for a wider examination of the Bay
which took place a few years after the Peace of Utrecht. In 1719,
Captain James Knight received orders from the Company to fit out
an expedition and sail up the west coast of the Bay. This he did in
two ships, the Albany frigate, Captain George Barlow, and the
Discovery, Captain David Vaughan. Captain John Scroggs, in the ship
Whalebone, two years afterward, sailed up the coast in search of the
expedition. It is maintained by the opponents of the Company that
these attempts were a mere blind to meet the search for a North-
West Passage, and that the Company was averse to any real
investigation being made.
It is of course impossible to say whether this charge was deserved
or not. The fact that no practicable North-West Passage has ever
been discovered renders the arguments drawn from the running of
the tides, &c., of no value, and certainly justifies the Company to
some extent in its inaction. The fact that in 1736 the Hudson's Bay
Company yielded to the claim raised by Dobbs and his associates, is
to be noted in favour of the Company's contention that while not
believing in the existence of the North-West Passage, they were
willing to satisfy the excited mind of the English public. Their
expedition of the Churchill sloop, Captain Napper, and the Musquash
sloop, Captain Crow, accomplished nothing in solving the question in
dispute.
Disappointed with the efforts made by the Company at his request,
Dobbs, in 1737, took in hand to organize an expedition under
Government direction to go upon the search of the "Passage." At this
time he opened communication with Captain Christopher Middleton,
one of the best known captains in the service of the Hudson's Bay
Company. Middleton, being satisfied with the Company's service,
refused to leave it. Dobbs then asked him to recommend a suitable
man, and also arranged with Middleton to be allowed to examine the
records kept of his voyages, upon the Hudson's Bay Company ships.
This, however, came to nothing.
About 1740 Captain Middleton had cause to differ with the Company
on business matters, and entertained Dobbs' proposition, which was
that he should be placed in command of a British man-of-war and go
in search of the long-sought North-West Passage. Middleton gave
the Hudson's Bay Company a year's notice, but found them unwilling
to let him retire.
He had taken the step of resigning deliberately and adhered to it,
though he was disappointed in his command not being so
remunerative as he expected. In May, 1741, Captain Middleton
received his orders from the Lords of the Admiralty to proceed upon
his journey and to follow the directions given him as to finding a
North-West Passage. These had been prepared under Dobbs'
supervision. Directions are given as to his course of procedure,
should he reach California, and also as to what should be done in
case of meeting Japanese ships. Middleton was placed in charge of
Her Majesty's sloop the Furnace, and had as a companion and under
his orders the Discovery Pink, William Moore, Master. In due time,
Hudson Bay was reached, but in August the season seemed rather
late to proceed northward from "Cary's Swan's Nest," and it was
decided to winter in the mouth of Churchill River.
On July 1st, 1742, the expedition proceeded northward. Most
complete observations were made of weather, land, presence of ice,
natives of the coast, depth of bay, rivers entering bay, tides, and any
possible outlets as far as 88 deg. or 89 deg. W. longitude.
Observations were continued until August 18th, when the expedition
sailed home to report what it had found.
Captain Middleton read an important paper on "The Extraordinary
Degrees and Surprising Effects of Cold in Hudson Bay," before the
Royal Society in London.
No sooner had Middleton reached the Orkneys on his return voyage
than he forwarded to Dobbs, who was in Ireland, a letter and an
abstract of his journal. Lest this should have gone astray, he sent
another copy on his arrival in the Thames. The report was, on the
whole, discouraging as to the existence of a north-west passage.
Dobbs, however, was unwilling to give up his dream, and soon
began to discredit Middleton. He dealt privately with the other
officers of the ships, Middleton's subordinates, and with surprising
skill turned the case against Captain Middleton.
The case of Dobbs against Captain Middleton has been well stated
by John Barrow. Middleton was charged with neglect in having failed
to explore the line of coast which afforded a probability of a passage
to the north-west. The principal points at issue appear to have been
in respect to the following discoveries of Middleton, viz. the Wager
River, Repulse Bay, and the Frozen Strait. As regards the first, Mr.
Dobbs asserted that the tide came through the so-called river from
the westward; and this question was settled in the following year by
Captain Moore, who entirely confirmed Captain Middleton's report.
Repulse Bay, which well deserves the name it bears, was no less
accurately laid down by Captain Middleton, and of the Frozen Strait,
Sir Edward Parry remarks, "Above all, the accuracy of Captain
Middleton is manifest upon the point most strenuously urged against
him, for our subsequent experience has not left the smallest doubt
of Repulse Bay and the northern part of Welcome Bay being filled by
a rapid tide, flowing into it from the eastward through the Frozen
Strait."
Dobbs, by a high order of logic chopping, succeeded in turning the
case, for the time being, against Captain Middleton. Seldom has
greater skill been used to win a cause. He quotes with considerable
effect a letter by Sir Bibye Lake, addressed to the Governor of the
Prince of Wales Fort, Churchill River, reading: "Notwithstanding an
order to you, if Captain Middleton (who is sent ahead in the
Government's service to discover a passage north-west) should by
inevitable necessity be brought into real distress and danger of his
life and loss of his ship, in such case you are then to give him the
best assistance and relief you can." Dobbs' whole effort seems to be
to show that Middleton was hiding the truth, and this, under the
influence of his old masters, the Hudson's Bay Company. A copy of
Dobbs' Criticisms, laid before the Lords of the Admiralty, was
furnished Captain Middleton, and his answer is found in "Vindication
of the Conduct," published in 1743.
"An Account of the Countries adjoining to Hudson Bay" by Arthur
Dobbs, Esq., is a book published in the year after, and is really a
book of note. A quarto, consisting of upwards of 200 pages, it
showed a marvellous knowledge of colonization in America, of the
interior of the continent at that time, and incidentally deals with
Captain Middleton's journal. Its account of the journey of "Joseph La
France, a French Canadese Indian," from Lake Superior by way of
Lake Winnipeg to Hudson Bay, is the first detailed account on record
of that voyage being made. Evidently Arthur Dobbs had caught the
ear of the English people, and the Company was compelled to put
itself in a thorough attitude of defence.
Dobbs with amazing energy worked up his cause, and what a writer
of the time calls, "The long and warm dispute between Arthur
Dobbs, Esq. and Captain Middleton," gained much public notice. The
glamour of the subject of a north-west passage, going back to the
exploits of Frobisher, Baffin, and Button, touched the national fancy,
and no doubt the charge of wilful concealment of the truth made
against the Hudson's Bay Company, repeated so strenuously by
Dobbs, gained him adherents. Parliament took action in the matter
and voted 20,000l. as a reward for the discovery of a north-west
passage. This caused another wave of enthusiasm, and immediately
a subscription was opened for the purpose of raising 10,000l. to
equip an expedition for this popular enterprise. It was proposed to
divide the whole into 100 shares of 100l. each. A vigorous canvass
was made to secure the amount, and the subscription list bears the
names of several nobles, an archbishop, a bishop, and many
esquires. A perusal of the names suggests that a number of them
are Irish, and no doubt were obtained by Mr. Dobbs, who was often
at Lisburn in Ireland. The amount raised was 7,200l. The expedition,
we hear afterwards, cost upwards of 10,000l., but the money
needed was, we are told, willingly contributed by those who
undertook the enterprise. Mr. Dobbs, as was suitable, was a leading
spirit on the Committee of Management.
Two ships were purchased by the Committee, the Dobbs galley, 180
tons burden, Captain William Moore, and the California, 140 tons,
Captain Francis Smith. On May 24th, 1746, the two vessels,
provisioned and well fitted out for the voyage, left the mouth of the
Thames, being in company with the two ships of the Hudson's Bay
Company going to the Bay, the four ships being under the convoy of
the ship Loo, of forty guns, as France was at this time at war with
England. The voyage was rather prosperous, with the exception of a
very exciting incident on board the Dobbs galley. A dangerous fire
broke out in the cabin of the vessel, and threatened to reach the
powder-room, which was directly underneath, and contained "thirty
or forty barrels of powder, candles, spirits, matches, and all manner
of combustibles." Though, as the writer says, "during the
excitement, you might hear all the varieties of sea eloquence, cries,
prayers, curses, and scolding, mingled together, yet this did not
prevent the proper measures being taken to save the ship and our
lives."
The story of the voyage is given to us in a very interesting manner
by Henry Ellis, gentleman, agent for the proprietors of the
expedition. Though nearly one hundred pages are taken up with the
inevitable summaries of "The Several Expeditions to discover a
North-West Passage," yet the remaining portion of the book is well
written. After the usual struggle with the ice in Hudson Strait, as it
was impossible to explore southward during the first season, the
Dobbs galley and the California sailed for Port Nelson, intending to
winter there. They arrived on August 26th. Ellis states that they
were badly received by the Hudson's Bay officers at the first. They,
however, laid up their ships in Hayes River, and built an erection of
logs on the shore for the staff. The officers' winter quarters were
called "Montague House," named after the Duke of Montague,
patron of the expedition. After a severe winter, during which the
sailors suffered with scurvy, and, according to Ellis, received little
sympathy from the occupants of York Fort, the expedition left the
mouth of the Hayes River on June 24th, to prosecute their discovery.
After spending the summer coasting Hudson Bay and taking careful
notes, the officers of the vessels gladly left the inhospitable shore to
sail homeward, and the two ships arrived in Yarmouth Roads on
October 14th, 1747.
"Thus ended," says Ellis, "this voyage, without success indeed, but
not without effect; for though we did not discover a north-west
passage ... we returned with clearer and fuller proofs ... that
evidently such a passage there may be." It will be observed that Ellis
very much confirms Captain Middleton's conclusions, but Mr. Dobbs
no doubt made the best of his disappointment, and, as we shall see,
soon developed what had been from the first his real object, the
plan for founding a rival company.
CHAPTER IX.
THE INTERESTING BLUE-BOOK OF 1749.
"Le roi est mort"—Royalty unfavourable—Earl of Halifax—"Company asleep"—Petition to
Parliament—Neglected discovery—Timidity or caution—Strong "Prince of Wales"—
Increase of stock—A timid witness—Claims of discovery—To make Indians Christians—
Charge of disloyalty—New Company promises largely—Result nil.
Arthur Dobbs, Esq., was evidently worsted in his tilt with the
Hudson's Bay Company. His fierce onslaught upon Captain Middleton
was no doubt the plan of attack to enable him to originate the
expedition of the Dobbs galley and California. Even this voyage had
brought little better prospect of the discovery of a north-west
passage, except the optimistic words of Ellis, the use of which,
indeed, seemed very like the delectable exercise of "extracting
sunbeams from cucumbers."
But the energy of the man was in no way dampened. Indeed, the
indications are, as we survey the features of the time, that he had
strong backing in the governing circles of the country. Time was
when the Hudson's Bay Company basked in the sunshine of the
Court. It is, perhaps, the penalty of old institutions that as rulers
pass away and political parties change, the centre of gravity of
influence shifts. Perhaps the Hudson's Bay Company had not been
able to use the convenient motto, "Le Roi est mort: Vive le Roi!" At
any rate the strong Court influence of the Company had passed
away, and there is hardly a nobleman to be found on the list of
stockholders submitted by the Company to the Committee of the
Lords.
On the other hand, when Henry Ellis, the historian of the expedition,
writes his book in the year after his return, he is permitted to
dedicate it to His Royal Highness Frederick, Prince of Wales, is
privileged to refer in his dedication to a "gracious audience" allowed
him by the Prince after his return, and to speak of "the generous
care" expressed by the Prince "for the happy progress of his design."
Again, in a similar dedication of a book written four years afterwards
by Joseph Robson, a former employé of the Hudson's Bay Company,
but a book full of hostility to the Company, allusion is made to the
fact that the Earl of Halifax, Lord Commissioner of Trade and
Plantations, gave his most hearty approval to such plans as the
expedition sought to carry out. It is said of Lord Halifax, who was
called the Father of Colonies: "He knows the true state of the nation
—that it depends on trade and manufactures; that we have more
rivals than ever; that navigation is our bulwark and Colonies our
chief support; and that new channels should be industriously
opened. Therefore, we survey the whole globe in search of fresh
inlets which our ships may enter and traffic." Those familiar with the
work of Lord Halifax will remember that the great colonization
scheme by which Nova Scotia was firmly grappled to the British
Empire and the City of Halifax founded, was his; and the charge
made by Dobbs that for a generation the "Company had slept on the
shores of the Bay," would appeal with force to a man of such
energetic and progressive nature as the Lord Commissioner.
Accordingly, Dobbs now came out boldly; not putting the discovery
of the North-West Passage in the front of his plan, but openly
charging the Hudson's Bay Company with indolence and failure, and
asking for the granting of a charter to a rival company.
As summed up by the sub-committee to which the petition of Dobbs
and his associates was submitted, the charges were:—
I. The Company had not discovered, nor sufficiently attempted to
discover, the North-West Passage into the southern seas.
II. They had not extended their settlements to the limits given them
by their Charter.
III. They had designedly confined their trade within very narrow
limits:
(a) Had abused the Indians.
(b) Had neglected their forts.
(c) Ill-treated their own servants.
(d) Encouraged the French.
The Hudson's Bay Company, now put on their mettle, exhibited a
considerable amount of activity, and filed documents before the
Committee that in some respects met the charges against them.
They claimed that they had in the thirty years preceding the
investigation done a fair amount of exploratory work and discovery.
In 1719, they had sent out the Albany frigate and Discovery to the
northern regions, and neither of them returned to tell the tale. In
the same year its vessels on the Bay, the Prosperous and the
Success, one from York Factory, the other from Prince of Wales Fort,
had sailed up the coast on exploratory expeditions. Two years
afterward, the Prosperous, under Kelsey, made a voyage, and the
Success, under Captain Napper, had sailed from York Fort and was
lost. In the same year the Whalebone, under Captain John Scroggs,
went from England to Prince of Wales Fort, and after wintering
there, in the following year made a decided effort on behalf of the
Passage, but returned unsuccessful. In the year when Dobbs
became so persistent (1737) James Napper, who had been saved
from the wreck of the Success sixteen years before, took command
of the Churchill from Prince of Wales Fort, but on the exploration
died, and the vessel returned. The Musquash, under Captain Crow,
accompanied the Churchill, but returned with no hope of success.
This was the case presented by the Hudson's Bay Company. It was
still open to the opponents of the Company to say, as they did, that
the Hudson's Bay Company was not in earnest, wanted nothing done
to attract rivals, and were adepts in concealing their operations and
in hoodwinking the public.
A more serious charge was that they had not sought to reach the
interior, but had confined their trade to the shores of the Bay. Here it
seems that the opponents of the Company made a better case. It is
indeed unaccountable to us to-day, as we think that the Company
had now been eighty years trading on the Bay and had practically no
knowledge of the inheritance possessed by them. At this very time
the French, by way of Lake Superior, had journeyed inland, met
Indian tribes, traded with them, and even with imposing ceremonies
buried metal plates claiming the country which the Hudson's Bay
Company Charter covered as lying on rivers, lakes, &c., tributary to
Hudson Bay. It is true they had submitted instructions to the number
of twenty or thirty, in which governors and captains had been urged
to explore the interior and extend the trade among the Indian tribes.
But little evidence could be offered that these communications had
been acted on.
The chief dependence of the Company seems to have been on one
Henry Kelsey, who went as a boy to Hudson Bay, but rose to be chief
officer there. The critics of the Company were not slow to state that
Kelsey had been a refugee from their forts and had lived for several
seasons among the Indians of the interior. Even if this were so, it is
still true that Kelsey came to be one of the most enterprising of the
wood-runners of the Company. Dobbs confronted them with the fact
that the voyage from Lake Superior to Hudson Bay had been only
made once in their history, and that by Joseph La France, the
Canadian Indian. Certainly, whether from timidity, caution, inertia, or
from some deep-seated system of policy, it was true that the
Company had done little to penetrate the interior.
The charge that the Company abused the Indians was hardly
substantiated. The Company was dependent on the goodwill of the
Indians, and had they treated them badly, their active rivals, the
French, would simply have reaped the benefit of their folly. That the
price charged the Indians for goods was as large as the price paid
for furs was small, is quite likely to have been true. Civilized traders
all the world over, dealing with ignorant and dependent tribes, follow
this policy. No doubt the risks of life and limb and goods in remote
regions are great, and great profits must be made to meet them. It
is to be remembered, however, that when English and French
traders came into competition, as among the Iroquois in New York
State, and afterwards in the Lake Superior district, the quality of the
English goods was declared by the Indians better and their
treatment by the English on the whole more honest and aboveboard
than that by the French.
That traders should neglect their own forts seems very unlikely.
Those going to the Hudson Bay Main expected few luxuries, and
certainly did not have an easy life, but there was on the part of the
Company a vast difference in treatment as compared with that given
to the fur traders in New France as they went to the far west. No
doubt pressure for dividends prevented expenditure that was
unnecessary, but a perusal of the experience of Champlain with his
French fur company leads us to believe that the English were far the
more liberal and considerate in the treatment of employés.
The fortress of the River Churchill, known as the Prince of Wales
Fort, with its great ruins to be seen to-day, belonging to this period,
speaks of a large expense and a high ideal of what a fort ought to
be. During the examination of witnesses by the Committee, full
opportunity was given to show cases of ill-treatment of men and
poor administration of their forts. Twenty witnesses were examined,
and they included captains, merchants, and employés, many of
whom had been in the service of the Company on the Bay, but
whether, as Robson says, "It must be attributed either to their
confusion upon appearing before so awful an assembly, or to their
having a dependence on the Company and an expectation of being
employed again in their service," little was elicited at all damaging to
the Company.
The charge of the fewness of the forts and the smallness of the
trade was more serious. That they should have a monopoly of the
trade, and should neither develop it themselves, nor allow others to
develop it, would have been to pursue a "dog in the manger" policy.
They stated that they had on an average three ships employed
solely on their business, that their exports for ten years immediately
preceding amounted to 40,240l. and their imports 122,835l., which
they claimed was a balance of trade satisfactory to England.
The objection that the whole capital of the Company at the
commencement, 10,500l., was trifling, was perhaps true, but they
had made great profits, and they used them in the purchase of ships
and the building of forts, and now had a much more valuable
property than at the beginning. That they had been able to increase
their stock so largely was a tribute to the profits of their business
and to its ability to earn dividends on a greatly increased capital
stock.
The increase of stock as shown by the Company was as follows:—
Original stock £10,500
Trebled in 1690 31,500
Trebled in 1720 94,500
At this time there was a movement to greatly increase the stock, but
the stringency of the money market checked this movement, and
subscriptions of ten per cent. were taken, amounting to 3,150l. only.
This was also trebled and added to the original 94,500l., making a
total stock of 103,950l.
Some three years after the investigation by the Committee, one of
the witnesses, Joseph Robson, who gave evidence of the very
mildest, most non-committal character, appears to have received
new light, for he published a book called, "An Account of Six Years'
Residence in Hudson's Bay." He says in the preface, speaking of the
evidence given by him in the investigation, "For want of confidence
and ability to express myself clearly, the account I then gave was far
from being so exact and full as that which I intended to have given."
What the influence was that so effectually opened Robson's eyes, we
do not know. The second part of this work is a critique of the
evidence furnished by the Company, and from the vigour employed
by this writer as compared with the apathy shown at the
investigation, it is generally believed that in the meantime he had
become a dependent of Dobbs.
The plea put forward by the petitioners for the granting of a charter
to them contained several particulars. They had, at their own cost
and charges, fitted out two ships, the Dobbs galley and California, in
search of the North-West Passage to the West and Southern Ocean.
Their object was, they claimed, a patriotic one, and they aimed at
extending the trade of Great Britain. They maintained that though
the reward offered had been 20,000l., it was not sufficient to
accomplish the end, as they had already spent more than half of
that sum. Not withstanding this, they had discovered a number of
bays, inlets, and coasts before unknown, and inasmuch as this was
the ground of the Charter issued by Charles II. to the Hudson's Bay
Company, they claimed like consideration for performing a similar
service.
The petitioners made the most ample promise as to their future
should the charter be granted. They would persevere in their search
for the passage to the Southern Ocean of America, of which,
notwithstanding the frequent failures in finding it, they had a strong
hope. The forward policy of Lord Halifax of extensive colonization
they were heartily in favour of, and they undertook to settle the
lands they might discover. The question had been raised during the
investigation, whether the Company had done anything to civilize
the natives. They had certainly done nothing. Probably their answer
was that they were a trading company, and never saw the Indians
except in the months of the trading season, when in July and August
they presented themselves from the interior at the several factories.
The petitioners promised, in regard to the natives, that they would
"lay the foundation for their becoming Christians and industrious
subjects of His Majesty." Beyond the sending out of a prayer-book
from time to time, which seemed to indicate a desire to maintain
service among their servants, the Company had taken no steps in
this direction.
The closing argument for the bestowal of a charter was that they
would prevent French encroachments upon British rights and trade
on the continent of America. The petition makes the very strong
statement that the Hudson's Bay Company had connived at, or
allowed French and English to encroach, settle, and trade within
their limits on the south side of the Bay. Whatever may have been in
the mind of the petitioners on this subject of conniving with the
French, a perusal of the minutes of the Company fails to show any
such disposition. The Company in Charles II.'s times was evidently
more anti-French than the Government. They disputed the claim of
the French to any part of the Bay, and strongly urged their case
before the English Commissioners at the Treaty of Ryswick. One of
their documents, seemingly showing them to be im pressed with the
claim of priority of ownership of the French King, did propose a
division of the Bay, giving the south part of the Bay to the French
and the remainder to themselves. It is easy to understand a trading
company wishing peace, so that trade might go on, and knowing
that Hudson Bay, with its enormous coast line, afforded wide room
for trade, proposing such a settlement.
No doubt, however, the reference is to the great competition which
was, in a few years, to extend through the interior to the Rocky
Mountains. This was to be indeed a battle royal. Arthur Dobbs,
judging by his book, which shows how far ahead he was of his
opponents in foresight, saw that this must come, and so the new
Company promises to penetrate the interior, cut off the supply of
furs from the French, and save the trade to Britain. A quarter of a
century afterwards, the Hudson's Bay Company, slow to open their
eyes, perceived it too, and as we shall see, rose from their slumbers,
and entered the conflict.
The Report was made to the Privy Council, expressing appreciation
of the petition, and of the advanced views enunciated, but stating
that the case against the Hudson's Bay Company had not yet been
made out. So no new charter was granted!
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
textbookfull.com