Interactive Data Visualization for the Web 1st

Edition Scott Murray pdf download

https://ebookgate.com/product/interactive-data-visualization-for-
the-web-1st-edition-scott-murray/

Get Instant Ebook Downloads – Browse at https://ebookgate.com

Instant digital products (PDF, ePub, MOBI) available
Download now and explore formats that suit you...

Learning IPython for Interactive Computing and Data

Visualization 2nd Edition Cyrille Rossant

https://ebookgate.com/product/learning-ipython-for-interactive-
computing-and-data-visualization-2nd-edition-cyrille-rossant/

ebookgate.com

Data Visualization with Flash Builder Designing RIA and

AIR Applications with Remote Data Sources Visualizing the
Web Cesare Rocchi
https://ebookgate.com/product/data-visualization-with-flash-builder-
designing-ria-and-air-applications-with-remote-data-sources-
visualizing-the-web-cesare-rocchi/
ebookgate.com

Everyday Data Visualization Desiree Abbott

https://ebookgate.com/product/everyday-data-visualization-desiree-
abbott/

ebookgate.com

Learning Qlikview Data Visualization 1st Edition Karl

Pover

https://ebookgate.com/product/learning-qlikview-data-
visualization-1st-edition-karl-pover/

ebookgate.com
Web Cartography Map Design for Interactive and Mobile
Devices 1st Edition Ian Muehlenhaus (Author)

https://ebookgate.com/product/web-cartography-map-design-for-
interactive-and-mobile-devices-1st-edition-ian-muehlenhaus-author/

ebookgate.com

Handbook of Statistics 24 Data Mining and Data

Visualization C.R. Rao

https://ebookgate.com/product/handbook-of-statistics-24-data-mining-
and-data-visualization-c-r-rao/

ebookgate.com

Social Data Visualization with HTML5 and JavaScript Timms

https://ebookgate.com/product/social-data-visualization-with-
html5-and-javascript-timms/

ebookgate.com

HTML5 Graphing and Data Visualization Cookbook 1st Edition

Ben Fhala

https://ebookgate.com/product/html5-graphing-and-data-visualization-
cookbook-1st-edition-ben-fhala/

ebookgate.com

HTML5 graphing and data visualization cookbook learn how

to create interactive HTML5 charts and graphs with canvas
JavaScript and open source tools Ben Fhala
https://ebookgate.com/product/html5-graphing-and-data-visualization-
cookbook-learn-how-to-create-interactive-html5-charts-and-graphs-with-
canvas-javascript-and-open-source-tools-ben-fhala/
ebookgate.com
 Interactive Data Visualization
for the Web
Download from Wow! eBook <www.wowebook.com>

Scott Murray
Interactive Data Visualization for the Web
by Scott Murray
Copyright © 2013 Scott Murray. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/
institutional sales department: 800-998-9938 or corporate@oreilly.com.
Editor: Meghan Blanchette Indexer: Judith McConville
Production Editor: Melanie Yarbrough Cover Designer: Karen Montgomery
Copyeditor: Teresa Horton Interior Designer: David Futato
Proofreader: Linley Dolby Illustrator: Rebecca Demarest

March 2013: First Edition

Revision History for the First Edition:

2013-03-04: First release

2013-04-12: Second release

See http://oreilly.com/catalog/errata.csp?isbn=9781449339739 for release details.

Nutshell Handbook, the Nutshell Handbook logo, the cover image, and the O’Reilly logo are registered
trademarks of O’Reilly Media, Inc. Interactive Data Visualization for the Web, the cover image of a long-tail
bushtit, and related trade dress are trademarks of O’Reilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as
trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trade‐
mark claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and author assume no
responsibility for errors or omissions, or for damages resulting from the use of the information contained
herein.

ISBN: 978-1-449-33973-9
[LSI]
 Table of Contents

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Why Data Visualization? 1
Why Write Code? 2
Why Interactive? 2
Why on the Web? 3
What This Book Is 3
Who You Are 4
What This Book Is Not 5
Using Sample Code 5
Thank You 6

2. Introducing D3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
What It Does 7
What It Doesn’t Do 8
Origins and Context 9
Alternatives 10
Easy Charts 10
Graph Visualizations 12
Geomapping 12
Almost from Scratch 13
Three-Dimensional 13
Tools Built with D3 14

3. Technology Fundamentals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
The Web 15
HTML 17
Content Plus Structure 18

iii
 Adding Structure with Elements 19
Common Elements 20
Attributes 22
Classes and IDs 22
Comments 23
DOM 24
Developer Tools 24
Rendering and the Box Model 27
CSS 29
Selectors 29
Properties and Values 31
Comments 31
Referencing Styles 31
Inheritance, Cascading, and Specificity 33
JavaScript 35
Hello, Console 35
Variables 35
Other Variable Types 36
Arrays 36
Objects 37
Objects and Arrays 38
Mathematical Operators 40
Comparison Operators 41
Control Structures 41
Functions 43
Comments 44
Referencing Scripts 44
JavaScript Gotchas 45
SVG 49
The SVG Element 50
Simple Shapes 50
Styling SVG Elements 53
Layering and Drawing Order 54
Transparency 55
A Note on Compatibility 57

4. Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Downloading D3 59
Referencing D3 60
Setting Up a Web Server 61
Terminal with Python 61
MAMP, WAMP, and LAMP 62

iv | Table of Contents
 Diving In 62

5. Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Generating Page Elements 63
Chaining Methods 65
One Link at a Time 66
The Hand-off 67
Going Chainless 67
Binding Data 67
In a Bind 67
Data 68
Please Make Your Selection 72
Bound and Determined 73
Using Your Data 76
High-Functioning 77
Data Wants to Be Held 78
Beyond Text 79

6. Drawing with Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Drawing divs 81
Setting Attributes 82
A Note on Classes 83
Back to the Bars 83
Setting Styles 84
The Power of data() 85
Random Data 87
Drawing SVGs 89
Create the SVG 89
Data-Driven Shapes 90
Pretty Colors, Oooh! 92
Making a Bar Chart 92
The Old Chart 93
The New Chart 93
Color 98
Labels 101
Making a Scatterplot 103
The Data 103
The Scatterplot 104
Size 105
Labels 106

Table of Contents | v
 Next Steps 107

7. Scales. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Apples and Pixels 109
Domains and Ranges 110
Normalization 111
Creating a Scale 111
Scaling the Scatterplot 112
d3.min() and d3.max() 112
Setting Up Dynamic Scales 114
Incorporating Scaled Values 114
Refining the Plot 115
Other Methods 119
Other Scales 119

8. Axes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Introducing Axes 121
Setting Up an Axis 122
Cleaning It Up 123
Check for Ticks 126
Y Not? 127
Final Touches 128
Formatting Tick Labels 130

9. Updates, Transitions, and Motion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Modernizing the Bar Chart 133
Ordinal Scales, Explained 134
Round Bands Are All the Range These Days 136
Referencing the Ordinal Scale 136
Other Updates 137
Updating Data 137
Interaction via Event Listeners 138
Changing the Data 139
Updating the Visuals 139
Transitions 142
duration(), or How Long Is This Going to Take? 143
ease()-y Does It 144
Please Do Not delay() 145
Randomizing the Data 147
Updating Scales 150
Updating Axes 152
each() Transition Starts and Ends 153

vi | Table of Contents
 Other Kinds of Data Updates 161
Adding Values (and Elements) 161
Removing Values (and Elements) 166
Data Joins with Keys 169
Add and Remove: Combo Platter 174
Recap 176

10. Interactivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Binding Event Listeners 177
Introducing Behaviors 178
Hover to Highlight 179
Grouping SVG Elements 184
Click to Sort 185
Tooltips 188
Default Browser Tooltips 189
SVG Element Tooltips 190
HTML div Tooltips 192
Consideration for Touch Devices 195
Moving Forward 195

11. Layouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Pie Layout 198
Stack Layout 202
Force Layout 206

12. Geomapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

JSON, Meet GeoJSON 213
Paths 215
Projections 216
Choropleth 219
Adding Points 222
Acquiring and Parsing Geodata 226
Find Shapefiles 226
Choose a Resolution 226
Simplify the Shapes 228
Convert to GeoJSON 228

13. Exporting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Bitmaps 233
PDF 234

Table of Contents | vii

 SVG 235

A. Appendix: Further Study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
Download from Wow! eBook <www.wowebook.com>

viii | Table of Contents

 Preface

This is a book about programming data visualizations for nonprogrammers. If you’re

an artist or graphic designer with visual skills but no prior experience working with data
or code, this book is for you. If you’re a journalist or researcher with lots of data but no
prior experience working with visuals or code, this book is for you, too.
This book will introduce you to D3, a JavaScript-based tool for loading data into a web
page and generating visuals from that data. I assume that you have little or no pro‐
gramming experience. Or, perhaps you have programmed before, but D3 and data vis‐
ualization are bringing you to JavaScript for the first time, and you’ve heard bad things
about it. Well, JavaScript is a little weird, but it’s not as bad as you’ve heard, and every‐
thing is going to be all right. Please sit down and make yourself comfortable.
This book began as a series of tutorials posted on my website. At the time (January 2012),
there wasn’t much information on D3 available that was accessible to beginners. Very
quickly, I was getting hundreds, then thousands of page views a day—evidence that
interest in the field generally (and D3 specifically) was growing like gangbusters. If
you’ve read the tutorials, portions of the book will feel familiar, but there is a lot of new
material here, including many more examples, sneaky tips, and warnings of things to
avoid. Also, the book contains 78 percent more bad jokes.
Data visualization is an interdisciplinary field, which is just one reason it’s impossible
to document the breadth of skills needed in a single book. Fortunately, because the field
is exploding in popularity, there are many new titles to choose from, each of which
complements this one.
On design process:

• Designing Data Visualizations: Intentional Communication from Data to Display

by Noah Iliinsky and Julie Steele. O’Reilly Media, 2011.
• Data Visualization: A Successful Design Process by Andy Kirk. Packt Publishing,
2012.

ix
On visual design principles and techniques:

• The Functional Art: An Introduction to Information Graphics and Visualization by

Alberto Cairo. New Riders, 2012.
• Information Dashboard Design: The Effective Visual Communication of Data by
Stephen Few. O’Reilly Media, 2006.

On the practicalities of working with data:

• Bad Data Handbook: Mapping the World of Data Problems by Q. Ethan McCallum.
O’Reilly Media, 2012.
• Data Analysis with Open Source Tools: A Hands-On Guide for Programmers and
Data Scientists by Philipp K. Janert. O’Reilly Media, 2010.
• Python for Data Analysis: Agile Tools for Real World Data by Wes McKinney.
O’Reilly Media, 2012.

Conventions Used in This Book

The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width
Used for program listings, as well as within paragraphs to refer to program elements
such as variable or function names, databases, data types, environment variables,
statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values deter‐
mined by context.

This icon signifies a tip, suggestion, or general note.

This icon indicates a warning or caution.

x | Preface
Using Code Examples
This book is here to help you get your job done. In general, if this book includes code
examples, you may use the code in this book in your programs and documentation. You
do not need to contact us for permission unless you’re reproducing a significant portion
of the code. For example, writing a program that uses several chunks of code from this
book does not require permission. Selling or distributing a CD-ROM of examples from
O’Reilly books does require permission. Answering a question by citing this book and
quoting example code does not require permission. Incorporating a significant amount
of example code from this book into your product’s documentation does require per‐
mission.
We appreciate, but do not require, attribution. An attribution usually includes the title,
author, publisher, and ISBN. For example: “Interactive Data Visualization for the Web
by Scott Murray (O’Reilly). Copyright 2013 Scott Murray, 978-1-449-33973-9.”
If you feel your use of code examples falls outside fair use or the permission given above,
feel free to contact us at permissions@oreilly.com.

Safari® Books Online

Safari Books Online is an on-demand digital library that delivers ex‐
pert content in both book and video form from the world’s leading
authors in technology and business.
Technology professionals, software developers, web designers, and business and crea‐
tive professionals use Safari Books Online as their primary resource for research, prob‐
lem solving, learning, and certification training.
Safari Books Online offers a range of product mixes and pricing programs for organi‐
zations, government agencies, and individuals. Subscribers have access to thousands of
books, training videos, and prepublication manuscripts in one fully searchable database
from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Pro‐
fessional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John
Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT
Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technol‐
ogy, and dozens more. For more information about Safari Books Online, please visit us
online.

Preface | xi
How to Contact Us
Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.

1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at http://oreil.ly/interactive_data_visualiza
tion_web.
To comment or ask technical questions about this book, send email to bookques
tions@oreilly.com.
For more information about our books, courses, conferences, and news, see our website
at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments
My name may be on the cover, but as an author, I feel as though I am merely funneling
the wisdom of hundreds of other brilliant minds onto the page.
First and foremost, I must thank my wife Nora, not least for being the first to say “Hey,
you should turn those tutorials into a book.” Without her support and encouragement,
this project never would have happened.
Thanks also to Rosten Woo, with whom I collaborated on my first D3 project, for
reaching out and giving me a reason to finally dig into this new tool. Thanks to Joe
Golike for several early D3 debugging sessions around that time, and to Jen Lowe and
Sha Hwang for their reviews and feedback on the initial tutorials.
I am extremely grateful to Casey Reas, Dan Shiffman, Joshua Noble, and Noah Iliinsky
—not just for offering advice on the book-creation process, but also for their ground‐
breaking work in the spheres of art, design, code, and data. Their careers have greatly
influenced my own.

xii | Preface
In that vein, I should also thank Jan Kubasiewicz at MassArt’s Dynamic Media Insti‐
tute. Back in 2007, Jan encouraged me to check out something called Processing, which
eventually led me to a whole new career in code-driven arts, data visualization, and now
this book.
It has been a pleasure working with my editor, Meghan Blanchette, and everyone else
on the team at O’Reilly. Thanks to Meghan and her crew for shepherding this project
all the way through, from concept to an actual, physical, chunk of paper with words and
strange diagrams printed on it.
Special thanks to Mike Bostock, Jen Lowe, Anna Powell-Smith, and Daisy Vincent for
agreeing to tech review the book and sending incredibly valuable feedback. The final
product is vastly improved, thanks to their input. That said, if you find an error or
confusing code example, it is because they begged me to rewrite it, and I steadfastly
refused.
Mike gets an extra-special thanks for developing D3 in the first place. Without this
elegant piece of software, the community of data visualization practitioners wouldn’t be
quite as vibrant, enthusiastic, and standards-compliant as it is today.
Speaking of community, many other people—including Jérôme Cukier, Lynn Cherny,
Jason Davies, Jeff Heer, Santiago Ortiz, Kim Rees, Moritz Stefaner, Jan Willem Tulp,
and others who I have forgotten to mention—on the D3 list and in nearby orbits have
also directly contributed to my thinking and process. Thank you for your support. I am
lucky to get to collaborate with so many talented people.

Preface | xiii
 CHAPTER 1
Introduction

Why Data Visualization?

Our information age more often feels like an era of information overload. Excess
amounts of information are overwhelming; raw data becomes useful only when we apply
methods of deriving insight from it.
Fortunately, we humans are intensely visual creatures. Few of us can detect patterns
among rows of numbers, but even young children can interpret bar charts, extracting
meaning from those numbers’ visual representations. For that reason, data visualization
is a powerful exercise. Visualizing data is the fastest way to communicate it to others.
Of course, visualizations, like words, can be used to lie, mislead, or distort the truth. But
when practiced honestly and with care, the process of visualization can help us see the
world in a new way, revealing unexpected patterns and trends in the otherwise hidden
information around us. At its best, data visualization is expert storytelling.
More literally, visualization is a process of mapping information to visuals. We craft
rules that interpret data and express its values as visual properties. For example, the
humble bar chart in Figure 1-1 is generated from a very simple rule: larger values are
mapped as taller bars.

Figure 1-1. Data values mapped to visuals

1
More complex visualizations are generated from datasets more complex than the se‐
quence of numbers shown in Figure 1-1 and more complex sets of mapping rules.

Why Write Code?

Mapping data by hand can be satisfying, yet is slow and tedious. So we usually employ
the power of computation to speed things up. The increased speed enables us to work
with much larger datasets of thousands or millions of values; what would have taken
years of effort by hand can be mapped in a moment. Just as important, we can rapidly
experiment with alternate mappings, tweaking our rules and seeing their output re-
rendered immediately. This loop of write/render/evaluate is critical to the iterative pro‐
cess of refining a design.
Sets of mapping rules function as design systems. The human hand no longer executes
the visual output; the computer does. Our human role is to conceptualize, craft, and
write out the rules of the system, which is then finally executed by software.
Unfortunately, software (and computation generally) is extremely bad at understanding
what, exactly, people want. (To be fair, many humans are also not good at this challenging
task.) Because computers are binary systems, everything is either on or off, yes or no,
this or that, there or not there. Humans are mushier, softer creatures, and the computers
are not willing to meet us halfway—we must go to them. Hence the inevitable struggle
of learning to write software, in which we train ourselves to communicate in the very
limited and precise syntax that the computer can understand.
Yet we continue to write code because seeing our visual creations come to life is so
rewarding. We practice data visualization because it is exciting to see what has never
before been seen. It is like summoning a magical, visual genie out of an inscrutable data
bottle.

Why Interactive?
Static visualizations can offer only precomposed “views” of data, so multiple static views
are often needed to present a variety of perspectives on the same information. The
number of dimensions of data are limited, too, when all visual elements must be present
on the same surface at the same time. Representing multidimensional datasets fairly in
static images is notoriously difficult. A fixed image is ideal when alternate views are
neither needed nor desired, and required when publishing to a static medium, such as
print.
Dynamic, interactive visualizations can empower people to explore the data for them‐
selves. The basic functions of most interactive visualization tools have changed little
since 1996, when Ben Shneiderman of the University of Maryland first proposed a

2 | Chapter 1: Introduction
“Visual Information-Seeking Mantra”: overview first, zoom and filter, then details-on-
demand.
This design pattern is found in most interactive visualizations today. The combination
of functions is successful, because it makes the data accessible to different audiences,
from those who are merely browsing or exploring the dataset to those who approach
the visualization with a specific question in search of an answer. An interactive visual‐
ization that offers an overview of the data alongside tools for “drilling down” into the
details may successfully fulfill many roles at once, addressing the different concerns of
different audiences, from those new to the subject matter to those already deeply familiar
with the data.
Of course, interactivity can also encourage engagement with the data in ways that static
images cannot. With animated transitions and well-crafted interfaces, some visualiza‐
tions can make exploring data feel more like playing a game. Interactive visualization
can be a great medium for engaging an audience who might not otherwise care about
the topic or data at hand.

Why on the Web?

Visualizations aren’t truly visual unless they are seen. Getting your work out there for
others to see is critical, and publishing on the Web is the quickest way to reach a global
audience. Working with web-standard technologies means that your work can be seen
and experienced by anyone using a recent web browser, regardless of the operating
system (Windows, Mac, Linux) and device type (laptop, desktop, smartphone, tablet).
Best of all, everything covered in this book can be done with freely accessible tools, so
the only investment required is your time. And everything we’ll talk about uses open
source, web-standard technologies.
By avoiding proprietary software and plug-ins, you can ensure that your projects are
accessible on the widest possible range of devices, from typical desktop computers to
tablets and even phones. The more accessible your visualization, the greater your au‐
dience and your impact.

What This Book Is

This book is a practical introduction to merging three practices—data visualization,
interactive design, and web development—using D3, a powerful tool for custom, web-
based visualization.
These chapters grew out of my own process of learning how to use D3. Many people,
including myself, come to D3 with backgrounds in design, mapping, and data visuali‐
zation, but not programming and computer science.

Why on the Web? | 3

D3 has a bit of an unfair reputation for being hard to learn. D3 itself is not so complicated,
but it operates in the domain of the Web, and the Web is complicated. Using D3 com‐
fortably requires some prior knowledge of the web technologies with which it interacts,
such as HTML, CSS, JavaScript, and SVG. Many people (myself included) are self-taught
when it comes to web skills. This is great, because the barrier to entry is so low, but
problematic because it means we probably didn’t learn each of these technologies from
the ground up—more often, we just hack something together until it seems to work,
and call it a day. Yet successful use of D3 requires understanding some of these tech‐
nologies in a fundamental way.
Because D3 is written in JavaScript, learning to use D3 often means learning a lot about
JavaScript. For many datavis folks, D3 is their introduction to JavaScript (or even web
development generally). It’s hard enough to learn a new programming language, let
alone a new tool built on that language. D3 will enable you to do great things with
JavaScript that you never would have even attempted. The time you spend learning both
the language and the tool will provide an incredible payoff.
My goal is to reduce that learning time, so you can start creating awesome stuff sooner.
We’ll take a ground-up approach, starting with the fundamental concepts and gradually
adding complexity. I don’t intend to show you how to make specific kinds of visualiza‐
tions so much as to help you understand the workings of D3 well enough to take those
building blocks and generate designs of your own creation.

Who You Are

You may be an absolute beginner, someone new to datavis, web development, or both.
(Welcome!) Perhaps you are a journalist interested in new ways to communicate the
data you collect during reporting. Or maybe you’re a designer, comfortable drawing
static infographics but ready to make the leap to interactive projects on the Web. You
could be an artist, interested in generative, data-based art. Or a programmer, already
familiar with JavaScript and the Web, but excited to learn a new tool and pick up some
visual design experience along the way.
Whoever you are, I hope that you:

• Have heard of this new thing called the “World Wide Web”
• Are a bit familiar with HTML, the DOM, and CSS
• Might even have a little programming experience already
• Have heard of jQuery or written some JavaScript before
• Aren’t scared by unknown initialisms like CSV, SVG, or JSON
• Want to make useful, interactive visualizations

4 | Chapter 1: Introduction
 If any of those things are unknown or unclear, don’t fear. You might just want to spend
more time with Chapter 3, which covers what you really need to know before diving
into D3.

What This Book Is Not

That said, this is definitely not a computer science textbook, and it is not intended to
teach the intricacies of any one web technology (HTML, CSS, JavaScript, SVG) in depth.
In that spirit, I might gloss over some technical points, grossly oversimplifying impor‐
tant concepts fundamental to computer science in ways that will make true software
engineers recoil. That’s fine, because I’m writing for artists and designers here, not en‐
gineers. We’ll cover the basics, and then you can dive into the more complex pieces once
you’re comfortable.
Download from Wow! eBook <www.wowebook.com>

I will deliberately not address every possible approach to a given problem, but will
typically present what I feel is the simplest solution, or, if not the simplest, then the most
understandable.
My goal is to teach you the fundamental concepts and methods of D3. As such, this
book is decidedly not organized around specific example projects. Everyone’s data and
design needs will be different. It’s up to you to integrate these concepts in the way best
suited to your particular project.

Using Sample Code

If you are a mad genius, then you can probably learn to use D3 without ever looking at
any sample code files, in which case you can skip the rest of this section.
If you’re still with me, you are probably still very bright but not mad, in which case you
should undertake this book with the full set of accompanying code samples in hand.
Before you go any further, please download the sample files from GitHub.
Normal people will want to click the ZIP link to download a compressed ZIP archive
with all the files. Hardcore geeksters will want to clone the repository using Git. If that
last sentence sounds like total gibberish, please use the first option.
Within the download, you’ll notice there is a folder for each chapter that has code to go
with it:
chapter_04
chapter_05
chapter_06
chapter_07
chapter_08
…

What This Book Is Not | 5

Files are organized by chapter, so in Chapter 9 when I reference 01_bar_chart.html,
know that you can find that file in the corresponding location: d3-book/chap‐
ter_9/01_bar_chart.html.
You are welcome to copy, adapt, modify, and reuse the example code in these tutorials
for any noncommercial purpose.

Thank You
Finally, this book has been handcrafted, carefully written, and pedagogically fine-tuned
for maximum effect. Thank you for reading it. I hope you learn a great deal, and even
have some fun along the way.

6 | Chapter 1: Introduction
 CHAPTER 2
Introducing D3

D3—also referred to as D3 or d3.js—is a JavaScript library for creating data visualiza‐

tions. But that kind of undersells it.
The abbreviation D3 references the tool’s full name, Data-Driven Documents. The da‐
ta is provided by you, and the documents are web-based documents, meaning anything
that can be rendered by a web browser, such as HTML and SVG. D3 does the driving,
in the sense that it connects the data to the documents.
Of course, the name also functions as a clever allusion to the network of technologies
underlying the tool itself: the W3, or World Wide Web, or, today, simply “the Web.”
D3’s primary author is the brilliant Mike Bostock, although there are a few other dedi‐
cated contributors. The project is entirely open source and freely available on GitHub.
D3 is released under a BSD license, so you may use, modify, and adapt the code for
noncommercial or commercial use at no cost.
D3’s official home on the Web is d3js.org.

What It Does
Fundamentally, D3 is an elegant piece of software that facilitates generation and ma‐
nipulation of web documents with data. It does this by:

• Loading data into the browser’s memory

• Binding data to elements within the document, creating new elements as needed

7
 • Transforming those elements by interpreting each element’s bound datum and set‐
ting its visual properties accordingly
• Transitioning elements between states in response to user input

Learning to use D3 is simply a process of learning the syntax used to tell it how you
want it to load and bind data, and transform and transition elements.
The transformation step is most important, as this is where the mapping happens. D3
provides a structure for applying these transformations, but, as we’ll see, you define the
mapping rules. Should larger values make taller bars or brighter circles? Will clusters
be sorted on the x-axis by age or category? What color palette is used to fill in countries
on your world map? All of the visual design decisions are up to you. You provide the
concept, you craft the rules, and D3 executes it—without telling you what to do. (Yes,
it’s like the opposite of Excel’s pushy “Chart Wizard.”)

What It Doesn’t Do
Here is a list of things D3 does not do:

• D3 doesn’t generate predefined or “canned” visualizations for you. This is on pur‐

pose. D3 is intended primarily for explanatory visualization work, as opposed to
exploratory visualizations. Exploratory tools help you discover significant, mean‐
ingful patterns in data. These are tools like Tableau and ggplot2, which help you
quickly generate multiple views on the same data set. That’s an essential step, but
different from generating an explanatory presentation of the data, a view of the data
that highlights what you’ve already discovered. Explanatory views are more con‐
strained and limited, but also focused, and designed to communicate only the im‐
portant points. D3 excels at this latter step, but is not ideal for the former. (For ideas
on other tools, see the section “Alternatives” on page 10 later in this chapter.)
• D3 doesn’t even try to support older browsers. This helps keep the D3 codebase
clean and free of hacks to support old versions of Internet Explorer, for example.
The philosophy is that by creating more compelling tools and refusing to support
older browsers, we encourage more people to upgrade (rather than forestall the
process, thereby requiring us to continue to support those browsers, and so on—a
vicious cycle). D3 wants us to move forward.
• D3’s core functionality doesn’t handle bitmap map tiles, such as those provided by
Google Maps or Cloudmade. D3 is great with anything vector—SVG images or
GeoJSON data—but wasn’t originally intended to work with traditional map tiles.
(Bitmap images are made up of pixels, so resizing them larger or smaller is difficult
without a loss in quality. Vector images are defined by points, lines, and curves—
mathematical equations, really—and can be scaled up or down without a loss in
quality.) This is starting to change, with the introduction of the d3.geo.tile plug-

8 | Chapter 2: Introducing D3
 in. Prior to this plug-in, geomapping with D3 meant either going all-SVG and
avoiding tiles or using D3 to create SVG visuals on top of a base layer of map tiles
(which would be managed by another library, like Leaflet or Polymaps—see the
section “Alternatives” on page 10 later in this chapter). This question of how to in‐
tegrate bitmap tiles and vector graphics comes up a lot in the D3 community. As of
today, there is no super-simple and perfect answer, but I think you can expect to
see lots of work done in this area, and possibly the new tile-handling methods
integrated into the D3 core at some point in the future.
• D3 doesn’t hide your original data. Because D3 code is executed on the client side
(meaning, in the user’s web browser, as opposed to on the web server), the data you
want visualized must be sent to the client. If your data can’t be shared, then don’t
use D3. Alternatives include using proprietary tools (like Flash) or prerendering
visualizations as static images and sending those to the browser. (If you’re not in‐
terested in sharing your data, though, why would you bother visualizing it? The
purpose of visualization is to communicate the data, so you might sleep better at
night by choosing openness and transparency, rather than having nightmares about
data thieves.)

Origins and Context

The first web browsers rendered static pages; interactivity was limited to clicking links.
In 1996, Netscape introduced the first browser with JavaScript, a new scripting language
that could be interpreted by the browser while the page was being viewed.
This doesn’t sound as groundbreaking as it turned out to be, but this enabled web
browsers to evolve from merely passive browsers to dynamic frames for interactive,
networked experiences. This shift ultimately enabled every intrapage interaction we
have on the Web today. Without JavaScript, D3 would never exist, and web-based data
visualizations would be limited to prerendered, noninteractive GIFs. (Yuck. Thank you,
Netscape!)
Jump ahead to 2005, when Jeffrey Heer, Stuart Card, and James Landay introduced
prefuse, a toolkit for bringing data visualization to the Web. prefuse (spelled with all
lowercase letters) was written in Java, a compiled language, with programs that could
run in web browsers via a Java plug-in. (Note that Java is a completely different pro‐
gramming language than JavaScript, despite their similar names.)
prefuse was a breakthrough application—the first to make web-based visualization ac‐
cessible to less-than-expert programmers. Until prefuse came along, any datavis on the
Web was very much a custom affair.
Two years later, Jeff Heer introduced Flare, a similar toolkit, but written in ActionScript,
so its visualizations could be viewed on the Web through Adobe’s Flash Player. Flare,

Origins and Context | 9

like prefuse, relied on a browser plug-in. Flare was a huge improvement, but as web
browsers continued to evolve, it was clear that visualizations could be created with native
browser technology, no plug-ins required.
By 2009, Jeff Heer had moved to Stanford, where he was advising a graduate student
named Michael Bostock. Together, in Stanford’s Vis Group, they created Protovis, a
JavaScript-based visualization toolkit that relied exclusively on native browser technol‐
ogies. (If you have used Protovis, be sure to reference Mike’s introduction to D3 for
Protovis users.)
Protovis made generating visualizations simple, even for users without prior program‐
ming experience. Yet to achieve this, it created an abstract representation layer. The
designer could address this layer using Protovis syntax, but it wasn’t accessible through
standard methods, so debugging was difficult.
In 2011, Mike Bostock, Vadim Ogievetsky, and Jeff Heer officially announced D3, the
next evolution in web visualization tools. Unlike Protovis, D3 operates directly on the
web document itself. This means easier debugging, easier experimentation, and more
visual possibilities. The only downside to this approach is a potentially steeper learning
curve, but this book will make that as painless as possible. Plus, all the skills you gain
while learning about D3 will prove useful even beyond the realm of datavis.
If you’re familiar with any of these groundbreaking tools, you’ll appreciate that D3 de‐
scends from a prestigious lineage. And if you have any interest in the philosophy un‐
derlying D3’s elegant technical design, I highly recommend Mike, Vadim, and Jeff ’s
InfoVis paper, which clearly articulates the need for this kind of tool. The paper encap‐
sulates years’ worth of learning and insights made while developing visualization tools.

Alternatives
D3 might not be perfect for every project. Sometimes you just need a quick chart and
you don’t have time to code it from scratch. Or you might need to support older browsers
and can’t rely on recent technologies like SVG.
For those situations, it’s good to know what other tools are out there. Here is a brief,
noncomprehensive list of D3 alternatives, all of which use web-standard technologies
(mostly JavaScript) and are free to download and use.

Easy Charts
DataWrapper
A beautiful web service that lets you upload your data and quickly generate a chart
that you can republish elsewhere or embed on your site. This service was originally
intended for journalists, but it is helpful for everyone. DataWrapper displays in‐

10 | Chapter 2: Introducing D3
 teractive charts in current browsers and static images for old ones. (Brilliant!) You
can also download all the code and run it on your own server instead of using theirs.
Flot
A plotting library for jQuery that uses the HTML canvas element and supports
older browsers, even all the way back to Internet Explorer 6. It supports limited
visual forms (lines, points, bars, areas), but it is easy to use.
Google Chart Tools
Having evolved from their earlier Image Charts API, Google’s Chart Tools can be
used to generate several standard chart types, with support for old versions of IE.
gRaphaël
A charting library based on Raphaël (see later in this chapter) that supports older
browsers, including IE6. It has more visual flexibility than Flot, and—some might
say—it is prettier.
Highcharts JS
A JavaScript-based charting library with several predesigned themes and chart
types. It uses SVG for modern browsers and falls back on VML for old versions of
IE, including IE6 and later. The tool is free only for noncommercial use.
JavaScript InfoVis Toolkit
The JIT provides several preset visualization styles for your data. It includes lots of
examples, but the documentation is pretty technical. The toolkit is great if you like
one of the preset styles, but browser support is unclear.
jqPlot
A plug-in for charting with jQuery. This supports very simple charts and is great if
you are okay with the predefined styles. jqPlot supports IE7 and newer.
jQuery Sparklines
A jQuery plug-in for generating sparklines, typically small bar, line, or area charts
used inline with text. Supports most browsers, even back to IE6.
Peity
A jQuery plug-in for very simple and very tiny bar, line, and pie charts that supports
only recent browsers. Did I mention that this makes only very tiny visualizations?
+10 cuteness points.
Timeline.js
A library specifically for generating interactive timelines. No coding is required;
just use the code generator. There is not much room for customization, but hey,
timelines are really hard to do well. Timeline.js supports only IE8 and newer.

Alternatives | 11
YUI Charts
The Charts module for the Yahoo! User Interface Library enables creation of simple
charts with a goal of wide browser support.

Graph Visualizations
A “graph” is just data with a networked structure (for example, B is connected to A, and
A is connected to C).
Arbor.js
A library for graph visualization using jQuery. Even if you never use this, you should
check out how the documentation is presented as a graph, using the tool itself. (It’s
so meta.) It uses the HTML canvas, so it works only in IE9 or current browsers,
although some workarounds are available.
Sigma.js
A very lightweight library for graph visualization. You have to visit this website,
move your mouse over the header graphic, and then play with the demos. Sigma.js
is beautiful and fast, and it also uses canvas.

Geomapping
I distinguish between mapping (all visualizations are maps) and geomapping (visuali‐
zations that include geographic data, or geodata, such as traditional maps). D3 has a lot
of geomapping functionality, but you should know about these other tools.
Kartograph
A JavaScript-and-Python combo for gorgeous, entirely vector-based mapping by
Gregor Aisch with must-see demos. Please go look at them now. I promise you’ve
never seen online maps this beautiful. Kartograph works with IE7 and newer.
Leaflet
A library for tiled maps, designed for smooth interaction on both desktop and
mobile devices. It includes some support for displaying data layers of SVG on top

12 | Chapter 2: Introducing D3
of the map tiles. (See Mike’s demo “Using D3 with Leaflet”.) Leaflet works with IE6
(barely) or IE7 (better!) and of course all current browsers.
Modest Maps
The granddaddy of tiled map libraries, Modest Maps has been succeeded by Poly‐
maps, but lots of people still love it, as it is lightweight and works with old versions
of IE and other browsers. Modest Maps has been adapted for ActionScript, Pro‐
cessing, Python, PHP, Cinder, openFrameworks…yeah, basically everything. File
this under “oldie, but goodie.”
Polymaps
A library for displaying tiled maps, with layers of data on top of the tiles. Polymaps
relies on SVG and thus works best with current browsers.

Almost from Scratch

These tools, like D3, provide methods of drawing visual forms, but without predesigned
visual templates. If you enjoy the creative freedom of starting from scratch, you might
enjoy these.
Processing.js
A native JavaScript implementation of Processing, the fantastic programming lan‐
guage for artists and designers new to programming. Processing is written in Java,
so exporting Processing sketches to the Web traditionally involved clunky Java
applets. Thanks to Processing.js, regular Processing code can run natively, in the
browser. It renders using canvas, so only modern browsers are supported.
Paper.js
A framework for rendering vector graphics to canvas. Also, its website is one of the
most beautiful on the Internet, and their demos are unbelievable. (Go play with
them now.)
Raphaël
Another library for drawing vector graphics, popular due to its friendly syntax and
support for older browsers.

Three-Dimensional
D3 is not the best at 3D, simply because web browsers are historically two-dimensional
beasts. But with increased support for WebGL, there are now more opportunities for
3D web experiences.
PhiloGL
A WebGL framework specifically for 3D visualization.

Alternatives | 13
Three.js
A library for generating any sort of 3D scene you could imagine, produced by
Google’s Data Arts team. You could spend all day exploring the mind-blowing de‐
mos on their site.

Tools Built with D3

When you want to use D3 without actually writing any D3 code, you can choose one of
the many tools built on top of D3!
Crossfilter
A library for working with large, multivariate datasets, written primarily by Mike
Bostock. This is useful for trying to squeeze your “big data” into a relatively small
web browser.
Cubism
A D3 plug-in for visualizing time series data, also written by Mike Bostock. (One
of my favorite demos.)
Dashku
An online tool for data dashboards and widgets updated in real time, by Paul Jensen.
dc.js
The “dc” is short for dimensional charting, as this library is optimized for exploring
large, multidimensional datasets.
NVD3
Reusable charts with D3. NVD3 offers lots of beautiful examples, with room for
visual customizations without requiring as much code as D3 alone.
Polychart.js
More reusable charts, with a range of chart types available. Polychart.js is free only
for noncommercial use.
Rickshaw
A toolkit for displaying time series data that is also very customizable.
Tributary
A great tool for experimenting with live coding using D3, by Ian Johnson.

14 | Chapter 2: Introducing D3
 CHAPTER 3
Technology Fundamentals
Download from Wow! eBook <www.wowebook.com>

Solid familiarity with the following concepts will make your time with D3 a lot less
frustrating and a lot more rewarding. Consider this a brief refresher course on Web-
Making 101.

Beware! This is a pretty dense chapter, packed with years’ worth of web
development knowledge, and nothing in here is specific to D3. I rec‐
ommend skimming just the sections on information that is new to you,
and skipping the rest. You can always reference this chapter later as
questions arise.

The Web
If you’re brand new to making web pages, you will now have to think about things that
regular people blissfully disregard every day, such as this: How does the Web actually
work?
We think of the Web as a bunch of interlinked pages, but it’s really a collection of con‐
versations between web servers and web clients (browsers).
The following scene is a dramatization of a typical such conversation that happens
whenever you or anyone else clicks a link or types an address into your browser (mean‐
ing, this brief conversation is had about 88 zillion times every day):

CLIENT: I’d really like to know what’s going on over at somewebsite.com. I better call over
there to get the latest info. [Silent sound of Internet connection being established.]
SERVER: Hello, unknown web client! I am the server hosting somewebsite.com. What
page would you like?
CLIENT: This morning, I am interested in the page at somewebsite.com/news/.
SERVER: Of course, one moment.

15
 Code is transmitted from SERVER to CLIENT.
CLIENT: I have received it. Thank you!
SERVER: You’re welcome! Would love to stay on the line and chat, but I have other
requests to process. Bye!

Clients contact servers with requests, and servers respond with data. But what is a server
and what is a client?
Web servers are Internet-connected computers running server software, so called be‐
cause they serve web documents as requested. Servers are typically always on and always
connected, but web developers often also run local servers, meaning they run on the
same computer that you’re working on. Local means here; remote means somewhere
else, on any computer but the one right in front of you.
There are lots of different server software packages, but Apache is the most common.
Web server software is not pretty, and no one ever wants to look at it.
In contrast, web browsers can be very pretty, and we spend a lot of time looking at them.
Regular people recognize names like Firefox, Safari, Chrome, and Internet Explorer, all
of which are browsers or web clients.
Every web page, in theory, can be identified by its URL (Uniform Resource Locator) or
URI (Uniform Resource Identifier). Most people don’t know what URL stands for, but
they recognize one when they see it. By obsolete convention, URLs commonly begin
with www, as in http://www.calmingmanatee.com, but with a properly configured serv‐
er, the www part is wholly unnecessary.
Complete URLs consist of four parts:

• An indication of the communication protocol, such as HTTP or HTTPS

• The domain name of the resource, such as calmingmanatee.com
• The port number, indicating over which port the connection to the server should
be attempted
• Any additional locating information, such as the path of the requested file, or any
query parameters

A complete URL, then, might look like this: http://alignedleft.com:80/tutorials/d3/.

Typically, the port number is excluded, as web browsers will try to connect over port 80
by default. So the preceding URL is functionally the same as the following: http://
alignedleft.com/tutorials/d3/
Note that the protocol is separated from the domain name by a colon and two forward
(regular) slashes. Why two slashes? No reason. The inventor of the Web regrets the error.

16 | Chapter 3: Technology Fundamentals

HTTP stands for Hypertext Transfer Protocol, and it’s the most common protocol for
transferring web content from server to client. The “S” on the end of HTTPS stands for
Secure. HTTPS connections are used whenever information should be encrypted in
transit, such as for online banking or e-commerce.
Let’s briefly step through the process of what happens when a person goes to visit a
website.

1. User runs the web browser of her choice, then types a URL into the address bar,
such as alignedleft.com/tutorials/d3/. Because she did not specify a protocol, HTTP
is assumed, and “http://” is prepended to the URL.
2. The browser then attempts to connect to the server behind alignedleft.com across
the network, via port 80, the default port for HTTP.
3. The server associated with alignedleft.com acknowledges the connection and is
taking requests. (“I’ll be here all night.”)
4. The browser sends a request for the page that lives at /tutorials/d3/.
5. The server sends back the HTML content for that page.
6. As the client browser receives the HTML, it discovers references to other files needed
to assemble and display the entire page, including CSS stylesheets and image files.
So it contacts the same server again, once per file, requesting the additional infor‐
mation.
7. The server responds, dispatching each file as needed.
8. Finally, all the web documents have been transferred over. Now the client performs
its most arduous task, which is to render the content. It first parses through the
HTML to understand the structure of the content. Then it reviews the CSS selectors,
applying any properties to matched elements. Finally, it plugs in any image files and
executes any JavaScript code.

Can you believe that all that happens every time you click a link? It’s a lot more com‐
plicated than most people realize, but it’s important to understand that client/server
conversations are fundamental to the Web.

HTML
Hypertext Markup Language is used to structure content for web browsers. HTML is
stored in plain text files with the .html suffix. A simple HTML document looks like this:
<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>

HTML | 17
 <h1>Page Title</h1>
<p>This is a really interesting paragraph.</p>
</body>
</html>
HTML is a complex language with a rich history. This overview will address only the
current iteration of HTML (formerly known as HTML5) and will touch on only what
is immediately relevant for our practice with D3.

Content Plus Structure

The core function of HTML is to enable you to “mark up” content, thereby giving it
structure. Take, for example, this raw text:
Amazing Visualization Tool Cures All Ills A new open-source tool designed for
visualization of data turns out to have an unexpected, positive side effect:
it heals any ailments of the viewer. Leading scientists report that the tool,
called D3000, can cure even the following symptoms: fevers chills general
malaise It achieves this end with a patented, three-step process. Load in data.
Generate a visual representation. Activate magic healing function.
Reading between the lines, we can infer that this is a very exciting news story. But as
unstructured content, it is very hard to read. By adding structure, we can differentiate
between the headline, for example, and the body of the story.

Amazing Visualization Tool Cures All Ills

A new open-source tool designed for visualization of data turns out to have an unexpec‐
ted, positive side effect: it heals any ailments of the viewer. Leading scientists report that
the tool, called D3000, can cure even the following symptoms:

• fevers
• chills
• general malaise

It achieves this end with a patented, three-step process.

1. Load in data.
2. Generate a visual representation.
3. Activate magic healing function.

That has the same raw text content, but with a visual structure that makes the content
more accessible.
HTML is a tool for specifying semantic structure, or attaching hierarchy, relationships,
and meaning to content. (HTML doesn’t address the visual representation of a

18 | Chapter 3: Technology Fundamentals

document’s structure—that’s CSS’ job.) Here is our story with each chunk of content
replaced by a semantic description of what that content is.

Headline
Paragraph text

• Unordered list item

• Unordered list item
• Unordered list item

Paragraph text

1. Numbered list item

2. Numbered list item
3. Numbered list item

This is the kind of structure we specify with HTML markup.

Adding Structure with Elements

“Marking up” is the process of adding tags to create elements. HTML tags begin with <
and end with >, as in <p>, which is the tag indicating a paragraph of text. Tags usually
occur in pairs, in which case adding an opening and closing pair of tags creates a new
element in the document structure.
Closing tags are indicated with a slash that closes or ends the element, as in </p>. Thus,
a paragraph of text may be marked up like the following:
<p>This is a really interesting paragraph.</p>

Some elements can be nested. For example, here we use the em element to add emphasis.
<p>This is a <em>really</em> interesting paragraph.</p>

Nesting elements introduces hierarchy to the document. In this case, em is a child of p

because it is contained by p. (Conversely, p is em’s parent.)
When elements are nested, they cannot overlap closures of their parent elements, as
doing so would disrupt the hierarchy. For example:
<p>This could cause <em>unexpected</p>
<p>results</em>, and is best avoided.</p>

HTML | 19
Some tags never occur in pairs, such as the img element, which references an image file.
Although HTML no longer requires it, you will sometimes see such tags written in self-
closing fashion, with a trailing slash before the closing bracket:
<img src="photo.jpg" />
As of HTML5, the self-closing slash is optional, so the following code is equivalent to
the preceding code:
<img src="photo.jpg">

Common Elements
There are hundreds of different HTML elements. Here are some of the most common.
We’ll cover additional elements in later chapters. (Reference the excellent Mozilla De‐
veloper Network documentation for a complete listing.)
<!DOCTYPE html>
The standard document type declaration. Must be the first thing in the document.
html
Surrounds all HTML content in a document.
head
The document head contains all metadata about the document, such as its title
and any references to external stylesheets and scripts.
title
The title of the document. Browsers typically display this at the top of the browser
window and use this title when bookmarking a page.
body
Everything not in the head should go in the body. This is the primary visible content
of the page.
h1, h2, h3, h4
These let you specify headings of different levels. h1 is a top-level heading, h2 is
below that, and so on.
p
A paragraph!
ul, ol, li
Unordered lists are specified with ul, most often used for bulleted lists. Ordered
lists (ol) are often numbered. Both ul and ol should include li elements to specify
list items.
em
Indicates emphasis. Typically rendered in italics.

20 | Chapter 3: Technology Fundamentals

strong
Indicates additional emphasis. Typically rendered in boldface.
a
A link. Typically rendered as underlined, blue text, unless otherwise specified.
span
An arbitrary span of text, typically within a larger containing element like p.
div
An arbitrary division within the document. Used for grouping and containing re‐
lated elements.
Our earlier example could be given semantic structure by marking it up using some of
these element’s tags:
<h1>Amazing Visualization Tool Cures All Ills</h1>
<p>A new open-source tool designed for visualization of data turns out to have
an unexpected, positive side effect: it heals any ailments of the viewer.
Leading scientists report that the tool, called D3000, can cure even the
following symptoms:</p>
<ul>
<li>fevers</li>
<li>chills</li>
<li>general malaise</li>
</ul>
<p>It achieves this end with a patented, three-step process.</p>
<ol>
<li>Load in data.</li>
<li>Generate a visual representation.</li>
<li>Activate magic healing function.</li>
</ol>
When viewed in a web browser, that markup is rendered as shown in Figure 3-1.

HTML | 21
Figure 3-1. Typical default rendering of simple HTML

Notice that we specified only the semantic structure of the content; we didn’t specify
any visual properties, such as color, type size, indents, or line spacing. Without such
instructions, the browser falls back on its default styles, which, frankly, are not too
exciting.

Attributes
All HTML elements can be assigned attributes by including property/value pairs in the
opening tag.
<tagname property="value"></tagname>
The name of the property is followed by an equals sign, and the value is enclosed within
double quotation marks.
Different kinds of elements can be assigned different attributes. For example, the a link
tag can be given an href attribute, whose value specifies the URL for that link. (href is
short for “HTTP reference.”)
<a href="http://d3js.org/">The D3 website</a>

Some attributes can be assigned to any type of element, such as class and id.

Classes and IDs

Classes and IDs are extremely useful attributes, as they can be referenced later to identify
specific pieces of content. Your CSS and JavaScript code will rely heavily on classes and
IDs to identify elements. For example:

22 | Chapter 3: Technology Fundamentals

 <p>Brilliant paragraph</p>
<p>Insightful paragraph</p>
<p class="awesome">Awe-inspiring paragraph</p>
These are three very uplifting paragraphs, but only one of them is truly awesome, as I’ve
indicated with class="awesome". The third paragraph becomes part of a class of awe‐
some elements, and it can be selected and manipulated along with other class members.
(We’ll get to that in a moment.)
Elements can be assigned multiple classes, simply by separating them with a space:
<p class="uplifting">Brilliant paragraph</p>
<p class="uplifting">Insightful paragraph</p>
<p class="uplifting awesome">Awe-inspiring paragraph</p>

Now, all three paragraphs are uplifting, but only the last one is both uplifting and
awesome.
IDs are used in much the same way, but there can be only one ID per element, and each
ID value can be used only once on the page. For example:
<div id="content">
<div id="visualization"></div>
<div id="button"></div>
</div>

IDs are useful when a single element has some special quality, like a div that functions
as a button or as a container for other content on the page.
As a general rule, if there will be only one such element on the page, you can use an
id. Otherwise, use a class.

Class and ID names cannot begin with numerals; they must begin with
alphabetic characters. So id="1" won’t work, but id="item1" will. The
browser will not give you any errors; your code simply won’t work, and
you will go crazy trying to figure out why.

Comments
As code grows in size and complexity, it is good practice to include comments. These
are friendly notes that you leave for yourself to remind you why you wrote the code the
way you did. If you are like me, you will revisit projects only weeks later and have lost
all recollections of it. Commenting is an easy way to reach out and provide guidance
and solace to your future (and very confused) self.
In HTML, comments are written in the following format:
<!-- Your comment here -->

Anything between the <!-- and --> will be ignored by the web browser.

HTML | 23
DOM
The term Document Object Model refers to the hierarchical structure of HTML. Each
pair of bracketed tags (or, in some cases, a single tag) is an element, and we refer to
elements’ relative relationships to each other in human terms: parent, child, sibling,
ancestor, and descendant. For example, in this HTML:
<html>
<body>
<h1>Breaking News</h1>
<p></p>
</body>
</html>

body is the parent element to both of its children, h1 and p (which are siblings to each
other). All elements on the page are descendants of html.
Web browsers parse the DOM to make sense of a page’s content. As coders building
visualizations, we care about the DOM, because our code must navigate its hierarchy
to apply styles and actions to its elements. We don’t want to make all the div elements
blue; we need to know how to select just the divs of the class sky and make them blue.

Developer Tools
In the olden days, the web development process went like this:

1. Write some code in a text editor.

2. Save the files.
3. Switch to a browser window.
4. Reload the page, and see if your code worked.
5. If it didn’t work, take a guess at what went wrong inside the magical black box of
the web browser, then go back to step 1.

Browsers were notoriously secretive about what went on inside the rendering engine,
which made debugging a total nightmare. (Seriously, in the late 1990s and early 2000s,
I literally had nightmares about this.) Fortunately, we live in a more enlightened age,
and every modern-day browser has built-in developer tools that expose the inner work‐
ings of the beast and enable us to poke around under the hood (to mix incompatible
metaphors).
All this is to say that developer tools are a big deal and you will rely on them heavily to
both test your code and, when something breaks, figure out what went wrong.
Let’s start with the simplest possible use of the developer tools: viewing the raw source
code of an HTML page (see Figure 3-2).

24 | Chapter 3: Technology Fundamentals

Every browser supports this, although different browsers hide this option in different
places. In Chrome 23.0, it’s under View→Developer→View Source. In Firefox 17.0, look
under Tools→Web Developer→Page Source. In Safari 6.0, it’s under Develop→Show
Page Source (although you must first set the “Develop” menu to display under Sa‐
fari→Preferences→Advanced). Going forward, I’m going to assume that you’re using
the newest version of whatever browser you choose.

Figure 3-2. Looking at the source code in a new window

That gets you the raw HTML, but if any D3 or JavaScript code has been executed, the
current DOM may be vastly different.
Fortunately, your browser’s developer tools enable you to see the current state of the
DOM. And, again, the developer tools are different in every browser. In Chrome, find
them under View→Developer→Developer Tools. In Firefox, try Tools→Web Developer.
In Safari, first enable the developer tools (in Safari→Preferences→Advanced). Then, in
the Develop menu, choose Show Web Inspector. In any browser, you can also use the
corresponding keyboard shortcut (as shown adjacent to the menu item) or right-click
and choose “inspect element” or something similar.
Until recently, Safari and Chrome shared the same developer tools, but with Safari 6.0,
Apple completely redesigned their dev tools, much to the dismay of many web-
developing Safari fans. (The new tools are very hard to navigate, and I don’t think I’m
the only one who feels that way.) Whichever browser you use might look a bit different
from my screenshots, but the functionality will be very similar.

Developer Tools | 25
 Figure 3-3 shows the Elements tab of Chrome’s web inspector. Here we can see the
current state of the DOM. This is useful because your code will modify DOM elements
dynamically. In the web inspector, you can watch elements as they change.
Download from Wow! eBook <www.wowebook.com>

Figure 3-3. Chrome’s web inspector

If you look closely, you’ll already see some differences between the raw HTML and the
DOM, including the fact that Chrome generated the required html, head, and body
elements. (I was lazy and didn’t include them in my original HTML.)

26 | Chapter 3: Technology Fundamentals

One more thing: why am I focusing on Chrome, Firefox, and Safari? Why not Internet
Explorer, Opera, or the many other browsers out there? For one, it’s best to develop your
projects using a browser with the broadest support for web standards. Internet Explorer
made huge progress with versions 9 and 10, but Chrome, Firefox, and Safari are un‐
derstood to have the broadest standards support, and they are updated more frequently.
Second, you’re going to spend a lot of time using the developer tools, so you should
develop with a browser that has tools you enjoy using. I was pretty devoted to Safari
until the 6.0 update changed everything. Now I’m going back and forth between Chrome
and Firefox’s new dev tools. I recommend you try them all and decide what works best
for you.

Rendering and the Box Model

Rendering is the process by which browsers, after parsing the HTML and generating the
DOM, apply visual rules to the DOM contents and draw those pixels to the screen.
The most important thing to keep in mind when considering how browsers render
content is this: to a browser, everything is a box.
Paragraphs, divs, spans—all are boxes in the sense that they are two-dimensional rec‐
tangles, with properties that any rectangle can have, such as width, height, and positions
in space. Even if something looks curved or irregularly shaped, rest assured, to the
browser, it is merely another rectangular box.
You can see these boxes with the help of the web inspector. Just mouse over any element,
and the box associated with that element is highlighted in blue, as shown in Figure 3-4.

Rendering and the Box Model | 27

Figure 3-4. Inspector with element box highlighted

There’s a lot of information about the ul unordered list here. Note that the list’s total
dimensions (width and height) are shown in the yellow box at the element’s lower-left
corner. Also, the list’s position in the DOM hierarchy is indicated in the lower-left corner
of the inspector: html > body > ul.
The box for the ul expands to fill the width of the entire window because it is a block-
level element. (Note how under “Computed Style” is listed display: block.) This is in
contrast to inline elements, which rest in line with each other, not stacked on top of each
other like blocks. Common inline elements include strong, em, a, and span.
By default, block-level elements expand to fill their container elements and force any
subsequent sibling elements further down the page. Inline elements do not expand to
fill extra space, and happily exist side by side, next to their fellow inline neighbors.
(Discussion question: what kind of element would you rather be?)

28 | Chapter 3: Technology Fundamentals

CSS
Cascading Style Sheets are used to style the visual presentation of DOM elements. A
simple CSS stylesheet looks like the following:
body {
background-color: white;
color: black;
}
CSS styles consist of selectors and properties. Selectors are followed by properties, grou‐
ped in curly brackets. A property and its value are separated by a colon, and the line is
terminated with a semicolon, like the following:
selector {
property: value;
property: value;
property: value;
}
The same properties can be applied to multiple selectors at once by separating the se‐
lectors with a comma, as in the following:
selectorA,
selectorB,
selectorC {
property: value;
property: value;
property: value;
}

For example, you might want to specify that both p paragraphs and li list items should
use the same font size, line height, and color.
p,
li {
font-size: 12px;
line-height: 14px;
color: orange;
}
Collectively, this whole chunk of code (selectors and bracketed properties) is called a
CSS rule.

Selectors
D3 uses CSS-style selectors to identify elements on which to operate, so it’s important
to understand how to use them.
Selectors identify specific elements to which styles will be applied. There are several
different kinds of selectors. We’ll use only the simplest ones in this book.

CSS | 29
Other documents randomly have
different content
 Piracy and Rough Life on the River
It is not clear when Cave-in-Rock first became the headquarters
of the criminals who flourished on the Ohio, and preyed upon
primitive commerce and travel between Pittsburgh and the Lower
Mississippi. Shortly after the Revolution was under way, renegades
from eastern communities, corrupt stragglers from the American
army, and villains who had had their brutal training in western wilds,
began to seek in the Ohio valley refuge from the more orderly and
well settled communities. Samuel Mason, who had been an officer in
the Continental army, converted the cavern into an inn as early as
1797. While he occupied the Cave, and a few years thereafter, it was
known as “Cave-Inn-Rock.” It was ideally located. Every passing boat
must reveal itself to those in the Cave who had a long, clear view up
and down the river. A lookout could detect boats long before
boatmen could perceive the Cave. The bold beauty of the bluff made
it pleasant for the boats to run in near the sharply shelving shore,
and many travelers were thus simply and easily delivered into the
hands of the banditti. As an inn, where drink and rest could be had,
it decoyed them; as a scene for shrouded crime it was perfect.
The earliest travelers on the western rivers floated or propelled
themselves with paddles and oars in small, clumsy craft. The Indian
canoe or pirogue was heavy, but was managed with skill by those
accustomed to its use. With the growing stream of settlers and the
increasing number of settlements along the Ohio and Mississippi,
there arose a necessity for larger craft that would bear heavier
burdens. This brought the flatboat era covering the period from
1795 to 1820—that quarter of a century known as the Golden Age of
Flatboating. During that era river piracy was at its height. The lighter
boats, pirogues, skiffs, and batteaux were to the clumsy rafts and
flatboats bearing heavy cargoes what submarines and torpedo boats
have been to the heavier ships in later warfare. Inland piracy had its
advantage in using the small craft on dark nights for sudden
descents and escapes.
In the midst of this period the stately steamboat age began its
development. It was inaugurated in 1811 when the first steam-
propelled “water-walker” made its laborious and astonishing way
from Pittsburgh to New Orleans. By 1820 steamboats had become a
dependable factor in traffic, and were, to river travel, what the
railroad train was later to become to the slow stagecoach and freight
wagon. It was inevitable that under steamboat influence flatboats of
all types—arks, broadhorns, Orleans boats, keel-boats, and flat-
bottomed barges—would follow the primitive pirogues, skiffs, and
batteaux into retirement, except for neighborhood use. River piracy
waned with the conditions it preyed upon, but not until about 1830
did it cease utterly.
In society, as in nature, everything develops with opportunity
and disappears according to necessity. In the primitive age of river
craft many travelers were captured or killed by Indians bent on
revenge or pillage. These marauders were sometimes led by white
renegades. Later, pioneers floating down the Ohio or Mississippi on
flatboats came in contact with comparatively few savages, but were
exposed to a far more daring and dangerous enemy in the form of
river pirates—white men, many of them descendants of supposedly
civilized European families. These disappeared as the population
increased. Then ensued the reign of the more diplomatic river
pirates—the professional gamblers who, for a half century, used
cards and other gaming devices as instruments with which to rob
those who ventured into their society.
Such were the types of craft and men operating upon and
infesting the rivers in the early days. The country through which
these boats moved was not the country we see today. Changes in
the shapes and channels of the rivers have been numerous, only the
rock-defined reaches preserving their original contours. Appearances
in detail have greatly changed. The wonderful unbroken forests are
gone. Where they once stood are now fields and farms or cut-over
forests; every few miles there is a town. The river channels once
mysterious and uncertain are now carefully charted.
Early voyageurs going down the river had, of course, no guides
and there were no known marks to indicate their approach to any of
the features of the river as it wound through the wild, uninhabited
country. The boatmen who came afterwards carrying maps rudely
scratched, found them unsatisfactory because of inaccuracies or lack
of detail. Not until a handbook was made available, after some years
of careful compilation of river features, could the uninitiated navigate
2
the large rivers with any degree of safety.
The numerous charts in The Navigator show the curves, islands,
sandbars, eddies, and channels, and mark the location of towns and
many other places of significance. The accompanying text contains
instructions of value to the boatman, and historical data of interest.
It is curious, however, that no section of either the Mississippi or
Ohio is designated as one where outlaws were likely to be
encountered—not even Cave-in-Rock nor the mouth of Cache River,
which were long considered the most dangerous resorts on the Ohio.
In every edition of The Navigator about a page is devoted to a
description of the Cave and instructions to boatmen passing it, but
there is no reference to its grim history. Zadok Cramer was evidently
a practical man, with no eye to the speculative. It was not until 1814
that he added a few lines bearing on the Cave’s “economic” history:
“This cavern sometimes serves as a temporary abode for those
wanting shelter, in case of shipwreck, or other accident, which
happen on the river near it. Families have been known to reside here
tolerably comfortable from the northern blasts of winter. The mouth
of this cave was formerly sheltered, and nearly hid by some trees
growing in front of it, but the rude axe has leveled them to the earth
and the cavern is exposed to the open view of the passenger.
Emigrants from the states, twenty-seven years ago used to land here
and wagon their goods across the Illinois country, it not being more
than one hundred and twenty miles from this place to Kaskaskia on
the Mississippi.”
The Cave, of course, had more than criminal uses. How on one
occasion it served as a “temporary abode for those wanting shelter”
is recorded in The American Pioneer, published in 1842. In this
magazine Dr. Samuel P. Hildreth, under the title of “History of a
Voyage from Marietta to New Orleans in 1805,” gives an interesting
account of the schooner Nonpareil and her voyage south, based on
data furnished him by members of her crew. The boat was built at
Marietta and started down the river April 21, 1805. She was a sea-
going vessel intended to run on the lakes near New Orleans. The
captain doubtless steered his course by a copy of The Navigator. We
quote from Hildreth’s account of what the crew found in 1805 at the
well-known lair of outlaws:
“As the Nonpareil approached near the mouth of this dreaded
cave, a little after twilight, they were startled at seeing the bright
blaze of a fire at its entrance. Knowing of its former fame as the den
of a band of robbers, they could not entirely suppress the suspicion
it awoke in their minds of its being again occupied for the same
purpose. Nevertheless, as they had previously determined not to
pass this noted spot without making it a visit, they anchored the
schooner a little distance from the shore and landed in the skiff.
Being well armed with pistols they marched boldly up to the cavern
where, instead of being greeted with the rough language and
scowling visages of a band of robbers, they found the cave occupied
by smiling females and sportive children. A part of the women were
busily occupied with their spinning wheels, while others prepared the
evening meal. Their suspicions were not, however, fully removed by
all these appearances of domestic peace, still thinking that the men
must be secreted in some hidden corner of the cave ready to fall on
them unawares. On a little further conversation they found the
present occupants of the dreaded cave consisted of four young
emigrant families from Kentucky going to settle in Illinois. The
females were yet in the bloom of life. Their husbands had bought or
taken up lands a few miles back from the river, and after moving
their families and household goods to this spot had returned to their
former residences to bring out their cattle, in the meantime leaving
their wives and children in the occupancy of the cave till their return.
“Having brought, with their spinning wheels and looms, an
abundance of flax, the women spent the weary days of their
husbands’ absence in the useful employment of spinning. A large fire
in the mouth of the cave gave cheerfulness to the gloomy spot and
enabled them, at night, to proceed with their labors, while its bright
rays were reflected upon the looms, beds, and household utensils
which lay piled up along the side of the cave. By day the sun
afforded them light, the mouth of the cave being capacious and
elevated, while the roof sheltered them from the rain. They were in
daily expectation of the arrival of their husbands, when they would
move out on to their farms in company.
“A little conversation soon dissipated all suspicions of harm from
the minds of their visitors ... and, borrowing from them a torch, they
explored the hidden recesses of the cave. At this time no vestige of
its former occupants remained but a few scattered barrel staves, and
the traces of their fires against the blackened sides of the rock. The
walls, even at that early day, were thickly scored with the names of
former visitors, to which they hastily added their own, and
thousands have no doubt been added since. Bidding a warm farewell
to this singular and solitary community, they entered their boat,
greatly wondering at the courage and confidence of these lonely
females. Their surprise, however, in a manner subsided when they
reflected that they were the daughters of Kentucky and from the
land of Daniel Boone.”
The Nonpareil experienced no trouble with river pirates, but was
wrecked during a storm on the Mississippi and never reached her
proposed destination. So, in one form or another, every flatboat and
other early river craft suffered more or less trouble. History records
many robberies and other misfortunes, but its pages also show that,
notwithstanding the numerous trials and tribulations, early river life,
rough as it was, was more of a romance than a tragedy. Going down
the Ohio and Mississippi proved, in many instances, “easy sailing”
compared to the flatboatman’s overland trip north over the Natchez
Trace and other wilderness roads infested with highwaymen.
The usual plan of the river robbers was to station one or two of
their men and women at some prominent place on shore to hail a
passing boat. These decoys pleaded to be taken aboard, claiming
they were alone in the wilderness and wished to go to some
settlement further down the river, or that they desired to purchase
certain necessities which they lacked. If the boat was thus enticed
ashore, the crew saw their cargo unloaded, and plundered, or
beheld their craft continue its course down the river in the hands of
the enemy, themselves held as hostages or murdered.
Boat wreckers were another common source of great danger.
Under one pretext or another they managed to get aboard the boat
and scuttle it near a place where their confederates were prepared
to make an attack. Or, like Colonel Fluger, they waited until they
found a boat tied along the bank and then bored holes in the bottom
or dug out the caulking. When the ill-fated boat began to sink, the
fellow-wreckers rushed to the rescue and appropriated the goods for
their own use, killing part or all the crew if necessary.
Then, as now, a number of dangerous channels existed in the
Ohio and Mississippi. They were designated as such in The
Navigator. Near the head of some of them lived reliable settlers who
made it a business to pilot boats through for pay. Pirates frequently
succeeded in passing themselves off as trustworthy local pilots.
Boats turned over to such men for safe steering were usually
grounded and immediately thereafter delivered into the hands of
outlaws in waiting.
One of the dangerous channels, against which voyageurs were
warned by The Navigator, ran from the head of Walker’s Bar (a bar
beginning about two miles below Cave-in-Rock) down to Tower
Rock, and from there extended to the foot of Hurricane Island, a
total distance of about eight miles. The author of the river guide,
after devoting considerable space to directions for navigating this
channel and avoiding the Hurricane Bars, adds a suggestion: “Just
below the Cave, on the right bank, there is a person who is
sometimes employed to pilot boats through this serpentine channel,
and it is better for a stranger to pay a dollar or two for this purpose,
than run the risk of grounding on either one or the other of these
bars in low water. When the water is high there is no occasion for a
director.”
The outlaws at Cave-in-Rock turned to their advantage the
suggestion published in The Navigator. About ten miles above the
Cave, near Battery Rock, or on what has long since been called the
Jonathan Brown Old Place, the robbers stationed a man who offered
to pilot, for a small sum, single boats or small fleets through this
“serpentine channel.” He explained that the person referred to by
The Navigator as living “just below the Cave” was out on a visit and
would not return for a week or more. In the event the first man
failed, another, standing ready a few miles further down at Ford’s
Ferry, offered his services. The pilot who succeeded in being
employed grounded the boat in front of the Cave if, by the time he
reached the place, he judged the cargo was worth the risk and the
crew could be overpowered. If more time was required, he guided
the boat to the head of Hurricane Island. There it was either
wrecked or taken safely through the channel, the procedure
depending on whether or not he judged a profitable robbery
possible. Boatmen who declined to take a pilot aboard at Battery
Rock or Ford’s Ferry were likely, if the water was comparatively low,
to inquire for a director “just below the Cave.” The man procured
there, whether a member of the Cave band or not, invariably guided
the boat safely through. Thus by helping to maintain one reputable
and reliable place near the Cave for procuring the services of a pilot,
the robbers experienced little trouble in trapping the boats they
selected for that purpose.
Although most of the prospective victims were given little
consideration until after they had come within ten or twenty miles of
the Cave, in a number of instances the river pirates began setting a
trap for a boat long before it arrived at Shawneetown.
The fact that the victims were piloted to the Cave by certain
members of a band, or enticed into the place by some other means
for the sole purpose of robbery, is recorded by many early writers;
none of them, however, gives any details. All authors who touch on
the Cave’s history publish statements based on what other men and
women heard other people had experienced while in the hands of
the outlaws. Only one instance has been found in which the victim
himself (Dr. Charles H. Webb) recited to an author the details of how
he was decoyed to the Cave and how he escaped from the men then
occupying the place. The old flatboat robbers and flatboat wreckers
left no first-hand accounts of the methods they employed.
The year 1788 roughly marks the beginning of the big inflow of
settlers into the region west of the Alleghenies, also the beginning of
counterfeiting and other outlawry at Cave-in-Rock. Many travelers
and home-seekers followed the trails and went into the interior
afoot, on horseback, or in wagons; others took the river to some
river point and either settled there, or proceeded overland to an
inland section. Thus, by “long lines of wagons” and “great fleets of
boats” the middle West became settled. In the meantime many a
small party traveled alone over the trails or drifted down the river in
a single boat or in a small fleet, into the new and sparsely populated
country, and became easy prey for highway robbers or river pirates
3
who were likely to appear at any time and in any disguise.
The earliest connection of the Cave with the name of any outlaw
who became famous was in 1797, when Samuel Mason, of
Revolutionary fame and hideous fate, seems to have occupied it as a
main trap for his carefully worked out scheme of river piracy on a
large scale. He erected a great rude sign on the river bank near the
mouth of the Cave, proclaiming to every passerby that his “Liquor
Vault and House for Entertainment” was open to the public. Many
captains and their crews and many flatboat passengers were lured to
it. After Mason and his family left for the South, most of the
succeeding bands, during their necessarily short stay, operated a
gambling and drinking place on the same principle.
It was a common practice among outlaws frequently to change
not only their headquarters but their names. While at Cave-in-Rock
Mason was also known as “Wilson.” Thomas Ashe, who wrote about
it, probably did not know that the Wilson he described was Samuel
Mason. Among the various men who appeared after the departure of
Samuel Mason, alias “Wilson,” was one Jim Wilson. Whether Jim
Wilson was his real name is not known. However, between Samuel
Mason as “Wilson” and a later man known as “Jim Wilson” there has
been more or less confusion for almost a century, especially in
tradition. In 1897 William Courtney Watts wrote a historical
romance, Chronicles of a Kentucky Settlement, in which he presents
James Ford, of Ford’s Ferry notoriety, as “James Wilson.” James Ford
was in no way connected with Mason or with Wilson, but his
presentation under the fictitious name of “James Wilson” had added
to the already existing confusion.
After James Ford’s death, which occurred in 1833—and many
years before Watts applied the name of “James Wilson” to him—a
writer published a sketch of the career of one Jim Wilson at the
Cave. This sketch is here recapitulated, not as a story that can be
verified in all its details by history, but as a semi-historical tale which
may convey a better idea of the methods, life, and fate of the Cave’s
outlaws than formal history. Only one who will make a study of the
Cave’s past—from the available authenticated records down to some
of its absurd traditions—will recognize this story as a picture in which
facts fairly divide the scene with fiction, and painted in colors that
bring joy to the hearts of readers of dime novels. When and by
whom it was written or first published has not been ascertained. It
apparently was not written before 1836, for the author, in his
introduction, attempts a description of the Cave as it appeared that
year. The writer evidently had read Thomas Ashe’s account
published in 1808, and was also familiar with some of the Cave’s
printed history and oral traditions. The story was probably first
published in an old magazine or newspaper. In 1893 it appeared,
anonymously and without credit, in the Crittenden Press, of Marion,
Kentucky. From that weekly it was copied by many newspapers in
the lower Ohio Valley, and is now preserved, under various titles, in
many a scrap book.
This old story is interesting because it was written when stories
of the Cave were still fresh. Inaccuracies and confusions of names
and dates may have crept in, but it remains the first concise and
inherently reasonable account of how the Cave was first occupied as
a den by river criminals. In the presentation of the usual method of
the Cave’s renegades, it matters very little whether the first of those
desperate captains of crime bore the name of Wilson, Mason, or
Harpe. In this case it seems clearly the story of Samuel Mason about
1797. The names they assumed might vary with every flatboat or
raft that passed. An alias is ever the shield of the criminal. The story
describes not only a method actually employed by the Cave’s
outlaws for many years, but also a method by which the career of
more than one of these river pirates was, as we shall see later, so
tragically terminated. The story runs, as follows:
“About the year 1809, one Jim Wilson, a flatboatman, while
passing down the Ohio, was overtaken by a terrific storm. He
steered his boat under the shelter of a cliff. On landing he observed
the opening of the cave. He was attracted by the commodious
rooms with dry ceilings and sanded floors, and resolved that on his
return to Pittsburgh he would bring his family hither.
“In the following spring Wilson’s boat again landed at the foot of
the cliff. This time he was not alone, but with him came his wife, five
children, two slaves, and William Hall, the great counterfeiter. His
boat was loaded with provisions, stores, liquors, and arms, which he
had stolen from the government warehouse at Fort Pitt on the night
before his departure. The great cave was soon transformed into a
dwelling and tavern large enough to accommodate several travelers.
“Wilson’s object for landing and establishing himself in so remote
and romantic headquarters will be seen hereafter. A sign was
planted at the water’s edge bearing these words: ‘Wilson’s Liquor
Vault and House for Entertainment.’ This novel sign had a magnetic
effect upon the boatmen who were almost daily passing en route to
southern markets, with flatboats loaded with produce. The boat
crews were generally jovial fellows, fond of rum, rest, and
merriment, and hardly a boat passed without stopping. Many were
the guests at Wilson’s Tavern; thieves and gamblers stopped off here
and in a few months the place became infamous for its
licentiousness and blasphemy.
“Wilson had been for many years a deep-dyed criminal and only
came here that he might vary his crimes, and have a wider field for
operation. Out of his guests he soon formed a band of the most
noted robbers, murderers, and counterfeiters that, for two years,
had no parallel in modern history. Their headquarters were at the
Cave, but they had many stations along the Ohio above and below,
which were maintained for the purpose of preventing suspicion
being cast upon the genial landlord at the Cave. The principal station
was at Hurricane Island, where forty-five men were stationed all the
time.
“Each boat that landed at the Cave was captured and such of
the crew as would not join Wilson’s Gang were allowed to drift on to
Hurricane Island where they were again captured and the remainder
of the crew foully murdered and their bodies cast into the Ohio. With
new pilots and crews the boats and cargoes were taken to New
Orleans, and converted into cash which was conveyed to the Cave
through the wilderness of Kentucky and Tennessee.
“Many boats loaded with valuable cargoes left port on the upper
Ohio and its tributaries, under the guidance of experienced and
trustworthy officers. The officers and crews never returned. No
returns for sales were ever received. It soon became a mystery that
so many honorable men never came back to pay over the proceeds,
and to tell the perils of their voyage. It was many months before any
serious suspicions were created. After that it was found that the
cargoes were disposed of by entirely different crews from those
entrusted with them. There was but limited postal or other
communication in those days—letters of special importance were
carried by messengers who often fell into the hands of Wilson’s men.
Thereby they kept posted and, by changing the communication to
suit their purposes, and forwarding them by different carriers, often
thwarted the attempt of justice, and kept their whereabouts
enveloped in mystery for many months. ‘But it is a long lane that
hath no turn.’ It was finally ascertained that no tidings could be had
of any boat after it had passed certain points on the Ohio near
Wilson’s Tavern.
“A meeting of the Pittsburgh shippers was called and it was
determined to ferret out the mystery. This would be a shrewd piece
of detective work which would be attended by many dangers. A
large reward was offered for information as to the exact location of
the robber band. John Waller, a determined and ambitious man of
Maysville, Kentucky, resolved to secure the reward or perish in the
attempt. He was furnished with a cargo contributed by various
shippers along the Ohio, and with five trusted companions he set out
early in the spring of 1810. They floated with the current many days.
At last one evening they came in sight of the Cave, and were
attracted by the novel sign and also the presence of several females
on the bank, who made gestures for them to land. They held a hasty
consultation and resolved to land; a few sweeps of the steering oar
brought them to the foot of the cliff.”
That which follows this clear description of ordinary
circumstances is evidently a mixture of fact and fiction that
represents the imaginative style of the day. It is quite plain that the
author himself had not personally visited the Cave, but had relied
upon the fictions of Thomas Ashe or the reflections from Ashe’s
account that had gained circulation and belief. He accepts the
mythical “upper cave” and has the Cave divided off into rooms and a
“council chamber,” no relics of which have ever been reported by any
matter-of-fact observer from that time to this. The leader, “Jim
Wilson,” he converts into a semi-savage with matted and tangled
hair and beard, who is yet a shrewd trader and an orator of no mean
power for his day. On the occasion of the initiation of new recruits
Jim Wilson delivers a romantic and argumentative speech that is
equal to the best fiction of the times.
The story narrates graphically how Waller and his men were
overawed and compelled, under fear, to agree to join the robber
band; how they were received into it with melodramatic ceremonies
and then were oath-bound, but not fully trusted; how they made
their escape—the savage and astute robbers being, of course, fooled
for the exigencies of the event; how the Waller force combined with
its waiting reinforcements, returned, captured Jim Wilson and then
went to Hurricane Island and destroyed that part of the band; and
how eventually “Jim Wilson’s head was severed, his body buried ...
the head identified and delivered to the proper authorities at
Pittsburgh ... and the captors received the merited reward.” This last
point is plainly an echo of Mason’s fate.
This story of the activities of the early renegades of civilization,
and of the river pirates who occupied the Cave bears upon its face
the stamp of truth that fits neatly into practically all traditions from
about 1795 to about 1820.
Before Mason became famous, however, greater scoundrels than
he were to attract public attention, and hold it for some years. The
story of the Harpes—“Big” and “Little” Harpe—is one that may freeze
the blood as read now in the light of old records and personal
accounts that seem to bring the reader into the very presence of
these two brutes. In the security of law and order in these days the
facts seem remote, but when the sparse settlement of the West in
1799 is realized, and the further fact that wilderness hospitality
opened doors to all travelers and admitted these monsters freely
with good people, it is possible then to conceive the horror their
deeds and presence aroused.
 The Harpes—A Terrible Frontier
Story
4
The career of the two Harpes in Tennessee, southern Illinois,
and Kentucky, particularly Kentucky, at the close of the eighteenth
century has rarely been equalled in the history of crime, either in
peace or war. Its beginning was so sudden, its motives wrapped in
such mystery, its race so swift, and its circumstances so terrible and
unbelievably brutal as to justify Collins, the distinguished historian of
Kentucky, in referring to the brothers as “the most brutal monsters
of the human race.”
At that time, 1798–99, Kentucky had a pioneer population of
about two hundred thousand, which was largely centered in the new
trading and agricultural towns in the eastern part and in the rich
bluegrass country. The remainder of the state, except along the
water courses, was well nigh a wilderness. In the southern and
western portions buffalo grazed, and bear were plentiful. East
Tennessee, where the scourge of crime began, was even more
sparsely settled. This pioneer population was vigorous, rude, and
accustomed even to Indian atrocities. Among the settlers were many
who, as fugitives from justice, had deliberately sought seclusion from
the eastern states because of criminal offenses. The Ohio River was
infested with inland pirates, and the early rivermen themselves were
a rough and violent type. Isolation led well-meaning pioneers to be
generous and confiding to those whom they had tested, but to a
great degree might was right, and strangers looked askance at each
other and were prepared for the worst.
 Yet such a rude and hardy people as these were gripped with
horror at the atrocities of the Harpes, at their often unmeaning and
unprovoked murders. It is difficult in these days of well ordered
government to realize the mysterious terror and excitement that
began near Knoxville in 1798 and swept through the wilderness to
the borders of the Mississippi, and across the Ohio into Illinois like
some sudden, creeping fire that breaks out in underbrush, and
grows steadily in intensity and rage until it sweeps forests before it.
All this was, in a measure, realized in the breasts of human beings
as the hideous crimes of the Harpes increased.
Aside from the wars and the recorded importances of political
development, the episode of the Harpes is the most astounding
event in the early life of the Middle West. It engaged the memory of
men for forty years, and the pens of numerous historians, and
writers of memoir have been occupied with it ever since. In the main
the story has been well preserved, but in the details there has been
the variation that grows with repetition. The most dignified
historians have not disdained to seek the minute details attaching to
the persons and actions of these two men from the moment they
began their criminal career to the thrilling blood-chase in which the
older brother was captured and killed, and the younger escaped into
exile and to an even more dramatic and terrible death.
To this day the story of “The Harpes” and “Harpe’s Head” is told
about firesides in the Cave-in-Rock country, in southern and western
Kentucky and in eastern Tennessee. It has been perpetuated in folk
ballads and written by scores of pens. [93]
It is the purpose here to bring together the many threads of the
tale as they have been verified and corrected by original records
sought from Wisconsin to New Orleans, and from Knoxville to Cave-
in-Rock and the Mississippi River.
Judge James Hall, while living in Illinois, wrote a brief account of
one of the crimes committed by these outlaws, and in April, 1824,
published it in The Port Folio of Philadelphia. In his introductory
remarks he comments: “Neither avarice nor want nor any of the
usual inducements to the commission of crime, seemed to govern
their conduct. A savage thirst for blood—a deep-rooted enmity
against human nature, could alone be discovered in their actions....
Plunder was not their object; they took only what would have been
freely given them, and no more than what was necessary to supply
the immediate wants of nature; they destroyed without having
suffered injury, and without the prospect of benefit.... Mounted on
fine horses they plunged into the forest, eluded pursuit by frequently
changing their course, and appeared unexpectedly to perpetrate
new horrors, at points distant from those where they were supposed
to lurk.”
Judge Hall, up to that time, had done little more than describe
one of their last crimes, yet The Cincinnati Literary Gazette, May 28,
1825, came out with a statement admitting that there may have
been two outlaws by the name of Harpe, but added: “We have no
hesitation in asserting that their history, as published in The Port
Folio, is unworthy of belief.... The horrible details concerning these
men ... such disgusting sketches of human depravity and barbarism
manifest either a vitiated taste or a total disregard of the morals of
the community.”
As far as is now known, at least two papers published in the
month following came to the defense of Judge Hall’s account. The
Illinois Gazette, of Shawneetown, among other things, declared:
“The depravity and bloodshed which marked their existence ... are
circumstances too strongly impressed upon the recollections of our
early settlers to be contradicted at this date.”
The Columbian, of Henderson, Kentucky, in a half column article
devoted to the same subject, asserts: “The account published in The
Port Folio is correct in every essential point.... However it may be
regretted that such monsters as the Harpes ever should have existed
to disgrace humanity, yet it is an uncontrovertible fact.” [56]
In the August, 1825, issue of The Port Folio Judge Hall published
an account of another murder committed by the Harpes—the killing
of Thomas Langford, who was among their first victims in Kentucky.
In the same number he devotes a few pages to a verification of the
statements he published then and a few months previous. And
before half had been told about the Harpes, The Cincinnati Literary
Gazette was convinced of its error in doubting and disputing the
veracity of Judge Hall. Judge Hall wrote several pages justifying the
publication of the weird and wonderful facts of the career of the
Harpes. His arguments published in 1825 in his own defense hold
good today and may be equally well applied to the story of the
Harpes here given, which, as far as is known, is the first attempt to
compile a complete history of these notorious outlaws:
“If it is intended to be objected, that these ‘horrid details,’ even
if true, are not proper for publication—I reply, that whatever tends
to develop the history or character of a people, is a legitimate
subject of public discussion. History to be of any value must be true.
It must disclose not only the truth but the whole truth. In vain would
the historian seek this in the frail monuments vaguely preserved in
the uncertain legend of tradition. He must resort to national records
and to the testimony of writers contemporary with the events which
he attempts to describe, and if the latter abstain from the narration
of ‘disgusting sketches of human depravity and barbarism,’ history
must be curtailed of her most fruitful source of incident, and men
and nations stripped of their boldest peculiarity. It is perhaps
forgotten that ‘depravity and barbarism’ constitute almost the sole
basis of history, tragedy, and the epic song; that kings and courts
are nothing without them; that they revel amid ‘the pomp, pride,
and circumstance of glorious war;’ and stand forth in bold relief in
every department of civil subordination. It is to be deplored that
such is the fact; but while crime and folly continue to predominate in
the affairs of men, they will be found to swell the pages of those
who attempt to exhibit correct pictures of human nature.
“In describing the American backwoodsmen, a class of men
peculiar to our country, I have thought it proper to introduce among
other authentic anecdotes the story of the Harpes. My object was to
display as well the extraordinary sufferings to which the earliest
emigrants to the western country were exposed, as the courage with
which they met and repelled those hardships.”
The Harpes were believed to be brothers. They were natives of
North Carolina. Micajah, known as Big Harpe, was born about 1768,
and Wiley, known as Little Harpe, was born about 1770. Their father
was said to have been a Tory who fought under the British flag at
King’s Mountain and took part in a number of other battles against
the colonists. Before the close of the Revolution and immediately
thereafter many of the Tories living in the south Atlantic colonies fled
toward the Mississippi. Those who still sympathized with the King of
England and continued to live in the “Old States” were, in most
sections, ostracized by their neighbors. It was to this class that the
parents of the Harpes belonged; and it was, therefore, in an
environment of hatred for and by neighbors that the two sons grew
up.
About the year 1840 Colonel G. W. Sevier, son of Governor John
Sevier, in an interview with Lyman C. Draper, the historian, stated
that Big Harpe, when asked shortly before he was killed why he had
committed so many crimes, answered that he had been badly
treated and consequently had become disgusted with all mankind.
[12G] The same statement is made by J. W. M. Breazeale, another
well-known early Tennesseean, who had lived in Knoxville the
greater part of his life and had investigated the careers of the
outlaws.
One writer attributes their acts of fiendish inhumanity to the fact
that they believed every man’s life, whether good, indifferent or bad,
was predestined and that the All Wise had foreordained for them a
hatred of humanity and a career of crime. [121] Draper, in his
“Sketch of the Harpes,” comments on the fact that “their tawny
appearance and dark curly hair betrayed a tinge of African blood
coursing through their veins.”
Criminologists may or may not agree as to the underlying cause
of the great thirst for blood possessed by the Harpes, but the fact
that they were the most savage and terrible characters in this period
of American history cannot be disputed.
About the year 1795 the two men, accompanied by Susan
Roberts and Betsey Roberts, left North Carolina for Tennessee.
Susan claimed to be the legal wife of Big Harpe, whereas Betsey
merely posed as such. Big Harpe, however, claimed both women as
his wives. The Harpes cared as little for the laws of matrimony as for
any other laws and the legality or illegality of anything they did was
a matter of indifference to them.
The two men and their women roamed in central Tennessee
about two years. Most of their time was spent with a few stray Creek
and Cherokee Indians who at the time were ostracized by their
tribes and were committing atrocities against their own people as
well as against the whites. The Harpes joined the savages in their
outrages, and not only encouraged them in their bloody deeds, but
gave them many demonstrations showing to what extent barbarity
could be practiced. Asleep or awake they were armed with
tomahawks and knives and never took a step from camp without a
gun. They were always prepared to shed blood for the satisfaction of
shedding it, or to resist arrest should any attempt be made to
capture them. They lived like man-eating animals. The women as
well as the men wore leather hunting shirts and moccasins made
from the untanned skin of animals they killed. They never wore hats
except in the coldest weather and then used the kind they
“whanged” together with deer skin thongs. [121]
Some time during 1797 the four left middle Tennessee for the
new settlement of Knoxville. While wandering toward the eastern
part of the state they met a young Methodist preacher named
William Lambuth, who was traveling through the wilderness alone.
They robbed him and among his belongings found a Bible. In turning
the leaves, looking for bank bills, Big Harpe discovered on the front
page, written in plain letters the names “William Lambuth” and
“George Washington.” Pointing to the name of the General, Harpe
remarked: “That is a brave and good man, but a mighty rebel
against the King.” The articles found in Lambuth’s possession
convinced the Harpes that he was a preacher, whereupon they
returned to him not only his Bible but also the gun, the little money,
and the horse they had taken. Then abruptly turning from him and
shouting, “We are the Harpes,” they quickly disappeared. This is
probably the only instance in the lives of the Harpes, after the
beginning of their murderous career, when they had anyone, old or
young, in their power, and showed less than a fiendish barbarity.
[121]
Obeying the principle that birds of a feather flock together, the
Harpes, it seems, were attracted toward the new settlement of
Knoxville. In March, 1798, James Weir, on his way from South
Carolina to Kentucky, spent a few days in the town. Writing of his
short stay there he says:
“In the infant town of Knox the houses are irregular and
interspersed. It was County Court day when I came. The town was
confused with a promiscuous throng of every denomination. Some
talked, some sang, and mostly all did profanely swear. I stood
aghast, my soul shrank back to hear the horrid oaths and dreadful
indignities offered to the Supreme Governor of the Universe, who
with one frown is able to shake them into non-existence. There was
what I never did see before, viz., on Sunday, dancing, singing, and
playing of cards, etc.... It was said by a gentleman of the
neighborhood that ‘the Devil is grown so old that it renders him
incapable of traveling and that he has taken up in Knoxville and
there hopes to spend the remaining part of his days in tranquility, as
he believes he is among his friends,’ but as it is not a good principle
to criticise the conduct of others, I shall decline it with this general
reflection, that there are some men of good principles in all places,
but often more bad ones to counterbalance them.” [109]
The Harpes doubtless felt they could better gratify their thirst for
blood in the vicinity of a settlement like Knoxville than in a wide
wilderness where subjects for their cruelty were too few. They found
a small tract of cleared land on Beaver Creek, about eight miles west
Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about books and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!

ebookgate.com