Download the Full Version of textbook for Fast Typing at textbookfull.

com

Splunk Developer s Guide 2nd Edition Kyle Smith

https://textbookfull.com/product/splunk-developer-s-
guide-2nd-edition-kyle-smith/

OR CLICK BUTTON

DOWNLOAD NOW

Download More textbook Instantly Today - Get Yours Now at textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

The Developer s Guide to Microsoft Azure Microsoft

https://textbookfull.com/product/the-developer-s-guide-to-microsoft-
azure-microsoft/

textboxfull.com

Beginning Power Apps The Non Developer s Guide to Building

Business Applications 2nd Edition Tim Leung

https://textbookfull.com/product/beginning-power-apps-the-non-
developer-s-guide-to-building-business-applications-2nd-edition-tim-
leung/
textboxfull.com

SQL Server 2016 Developer s Guide 1st Edition Dejan Sarka

https://textbookfull.com/product/sql-server-2016-developer-s-
guide-1st-edition-dejan-sarka/

textboxfull.com

Soft Skills The Software Developer s Life Manual 2nd

Edition John Sonmez

https://textbookfull.com/product/soft-skills-the-software-developer-s-
life-manual-2nd-edition-john-sonmez/

textboxfull.com
AWS Certified Developer Associate Guide Your one stop
solution to pass the AWS developer s certification 1st
Edition Vipul Tankariya
https://textbookfull.com/product/aws-certified-developer-associate-
guide-your-one-stop-solution-to-pass-the-aws-developer-s-
certification-1st-edition-vipul-tankariya/
textboxfull.com

Tom Smith s Cricket Umpiring and Scoring 2017 Code 2nd

Edition Laws Edition Tom Smith

https://textbookfull.com/product/tom-smith-s-cricket-umpiring-and-
scoring-2017-code-2nd-edition-laws-edition-tom-smith/

textboxfull.com

Implementing Splunk 7 D.

https://textbookfull.com/product/implementing-splunk-7-d/

textboxfull.com

Compact Key for Schools Teacher s Book 2nd Edition Smith

https://textbookfull.com/product/compact-key-for-schools-teacher-s-
book-2nd-edition-smith/

textboxfull.com

Women s Studies The Basics 2nd Edition Bonnie G Smith

https://textbookfull.com/product/women-s-studies-the-basics-2nd-
edition-bonnie-g-smith/

textboxfull.com
Splunk Developer's Guide
Second Edition

Learn the A to Z of building excellent Splunk

applications with the latest techniques using this
comprehensive guide

Kyle Smith

BIRMINGHAM - MUMBAI
Splunk Developer's Guide
Second Edition

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book is
sold without warranty, either express or implied. Neither the author, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages
caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

First published: May 2015

Second edition: January 2016

Production reference: 1190116

Published by Packt Publishing Ltd.

Livery Place
35 Livery Street
Birmingham B3 2PB, UK.

ISBN 978-1-78588-237-1

www.packtpub.com
 Credits

Author Project Coordinator

Kyle Smith Suzanne Coutinho

Reviewer Proofreader
Marco Scala Safis Editing

Commissioning Editor Indexer

Veena Pagare Hemangini Bari

Acquisition Editor Graphics

Vinay Argekar Abhinash Sahu

Content Development Editor Production Coordinator

Amey Varangaonkar Shantanu N. Zagade

Technical Editor Cover Work

Taabish Khan Shantanu N. Zagade

Copy Editor
Trishya Hajare
 About the Author

Kyle Smith is a self-proclaimed geek and has been working with Splunk
extensively since 2010. He enjoys integrating Splunk with new sources of data and
types of visualization. He has spoken numerous times at the Splunk User Conference
(most recently in 2014 on Lesser Known Search Commands) and is an active contributor
to the Splunk Answers community and also to the #splunk IRC channel. He was
awarded membership into the SplunkTrust as a founding member. He has published
several Splunk Apps and add-ons to Splunkbase, the Splunk community's premier
Apps and add-ons platform. He has worked in both higher education and private
industry; he is currently working as an integration developer for Splunk's longest
running professional services partner. He lives in central Pennsylvania with
his family.

I'd like to thank my wife who most graciously put up with all
my BS during the writing of this book. Without her, this effort is
meaningless.
 About the Reviewer

Marco Scala has been working for more than 15 years delivering solutions to large
enterprise customers, first in the APM and J2EE fields and, since 2009, in the fields
of operational intelligence and Splunk. He has provided consultancy for big Splunk
installations for major customers, focusing on the best and most effective solutions
for each different customer's needs. Since 2012, he's also a certified Splunk trainer.

In the last few years, Marco's major focus has been to get Splunk customers to
gain the maximum value from their IT data and provide the business a better view
and insight. Big Data is another major field of interest, and his next challenge is
using Splunk to give customers useful insights and a practical implementation and
exploitation of Big Data.
 www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files
available? You can upgrade to the eBook version at www.PacktPub.com and as a print book
customer, you are entitled to a discount on the eBook copy. Get in touch with us at service@
packtpub.com for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range
of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

TM

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library.
Here, you can search, access, and read Packt's entire library of books.

Why subscribe?
• Fully searchable across every book published by Packt
• Copy and paste, print, and bookmark content
• On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib
today and view 9 entirely free books. Simply use your login credentials for immediate access.

Instant updates on new Packt books

Get notified! Find out when new books are published by following @PacktEnterprise on
Twitter or the Packt Enterprise Facebook page.
 Table of Contents
Preface v
Chapter 1: Application Design Fundamentals 1
What is a Splunk application? 1
Why applications? 2
Definitions 2
Designing the App 4
Identifying the use case 5
Identifying what you want to consume 5
Identifying what you want to brand 6
Identifying what you want to display 6
Installing Apps 7
Splunk Web 7
The Splunk command line 8
Unzipping using the command line 8
Summary 9
Chapter 2: Creating Applications 11
A brief clarification 11
Methods of creating applications 12
GUI 13
CLI 15
FreeForm 15
Basic application structure 18
appserver 18
bin 19
default 19
local 19
lookups 19

[i]
Table of Contents

metadata 19
static 20
Application data 20
Indexes 20
Source types 23
Sources 23
Available Splunk knowledge objects 23
Macros 24
Event types 24
Tags 25
Saved searches 25
Dashboards 26
Lookups 26
Configurations 27
Object permissions 27
The setup screen 29
The endpoint 30
The setup file 31
Summary 32
Chapter 3: Enhancing Applications 33
Workflows 33
Custom alert actions 37
Enriched data 39
Event types 39
Tags 44
Macros 46
Lookups 48
Common Information Model 53
Branding your App 53
Logos 54
Navigation 56
CSS 58
JavaScript 58
Acceleration 58
Summary indexing 58
Accelerated reports 60
Summary 61

[ ii ]
 Table of Contents

Chapter 4: Basic Views and Dashboards 63

Knowing your data 63
Available modules 64
SimpleXML dashboard 65
SimpleXML forms 68
Custom JavaScript, CSS, and Tokens 72
HTML dashboards 73
Summary 75
Chapter 5: The Splunk Web Framework 77
The HTML dashboard 77
SplunkJS Stack 81
Search-related modules 82
SearchManager 82
SavedSearchManager 83
PostProcessManager 84
View-related modules 85
ChartView 85
Display-related modules 90
CheckboxView 91
CheckboxGroupView 91
DropdownView 92
EventsViewerView 93
FooterView 94
HeaderView 94
MultiDropdownView 95
RadioGroupView 95
SearchBarView 96
SearchControlsView 97
SimpleSplunkView 97
SingleView 97
MapElement 98
TableView 99
TextInputView 100
TimeRangeView 100
TimelineView 101
Tokenization 101
Customizing Splunk dashboards using CSS 102
Customizing Splunk dashboards using JavaScript 105
Custom D3 visualization 109
External data and content 112
Data 113
Content 113
Summary 114

[ iii ]
Table of Contents

Chapter 6: Advanced Integrations and Development 115

Modular D3 visualization 115
Modular inputs 122
The spec file 124
Testing modular inputs 129
Configuring modular inputs 129
The App Key Value Store 131
When would you use the KV Store? 131
Configuring the KV Store 133
Data models 136
Version control and package managers 140
npm 141
Bower 141
Gulp 142
Git 142
Tying them all together 142
Summary 146
Chapter 7: Packaging Applications 147
Naming guidelines 147
Dos and don'ts 150
Packaging the App 151
The App packaging checklist 154
Summary 155
Chapter 8: Publishing Applications 157
Self-hosting your App 157
Splunkbase 158
Certified Applications 160
Splunk Cloud applications 162
Community 162
Answers 163
dev.splunk.com 163
Internet Relay Chat 163
Wiki 163
User groups 164
The SplunkTrust 164
Summary 165
Index 167

[ iv ]
 Preface
Splunk is awesome. Not only can you consume virtually any data with it, you can
also extend and integrate Splunk with virtually any external system. Splunk uses sets
of configurations that are referred to as applications or add-ons, which is the primary
focus of this book. Leveraging these applications and add-ons is what gives Splunk
its unique ability to extend, learn, analyze, and visualize information.

Splunk helps users to determine the root cause of a failure, a quick overview of
system health, and dive deep into SQL statements and messages, just to name a
few. The aggregation and centralization of log and event management is a growing
trend in the Big Data space. By leveraging the combined intelligence gathered
from correlating disparate sets of data, businesses or individuals can make
data-based decisions. This book will help a Splunk developer, or even just a
curious end user, to develop different methods of consuming new data, design
new types of visualization, or even just offer tips and tricks that help the software
development lifecycle.

Overview of what this book isn't

Most developer guides will tell you what their book is and/or does. We aim to
explain what this book isn't, and allow you to fill in the rest with your imagination!
Thus, proceed to this list:

• Will not cover Splunk basics

• Will not cover creating dashboards via the GUI (other than HTML)
• Will not discuss how to code in Python
• Will not discuss statistics
• Will not cover SDKs
• Will not discuss making beer

[v]
Preface

Splunk basics will not be covered. These include concepts such as searching (finding
data, using timecharts, stats, some eval commands, and so on), reporting (making
basic pie charts or line charts via the GUI), data inputs (basic file monitoring, TCP
and UDP inputs, Splunk forwarders, and so on), and configurations (GUI and web-
based configuration editing), to name a few. Creating dashboards via the GUI?
Nope. Python will be discussed and sample code will be provided, but this book will
not cover the nuances of the code, nor will it teach you Python syntax. We will not
cover statistical computation, other than how to practically apply some basic math
to create value-based visualizations. We will not cover using the SDKs (software
development kits) being used in custom Splunk applications that are external to
Splunk (for example, Angular, PHP, .NET, and others). These are out of the scope of
this book. Free as in beer? Nope, the choice of hops, starch, and oak-barrel aging for
the creation of beer will not be discussed, but rather consumed during the writing
and/or reading of this book.

Unless otherwise stated, this book uses Splunk version 6.3 as the
development environment.

What this book is

This book will guide you through many the different areas of Splunk App and add-
on creation. We will start by looking at the design aspects of an App or add-on, how
to create them, what knowledge objects are available for use within the App, ways
to enhance your App with metadata and external data, and some basic views and
dashboards. From there, we will move into the Splunk Web Framework, modular
inputs, jQuery, web framework programs, and then packaging and publishing Apps
and add-ons. At the tail end, we will highlight some areas of the Splunk community
that prove to be very useful.

Assumptions
There are a few basic assumptions that we are going to make. Having purchased or
otherwise obtained this book, we assume that you are interested in developing with
Splunk, and have a basic understanding of Splunk and how to navigate around the
software. Knowledge of saving searches, reports, and basic dashboarding is a must,
since most concepts and examples will be built upon the basics. We also assume
that you have basic knowledge of HTML, CSS, JS, and some XML. Here, XML will
be limited to the Splunk XML framework specifically. We would also recommend
you to have knowledge of, or proficiency in, Python, RequireJS, and other web
technologies such as Bower, npm, and Gulp. We will demonstrate how to use these
web technologies within a Splunk application.
[ vi ]
 Preface

What this book covers

Chapter 1, Application Design Fundamentals, discusses fundamental questions and
considerations before diving into an App or add-on configuration.

Chapter 2, Creating Applications, discusses the basic methods of App and add-on
creation, along with an explanation of the structure of an App or add-on.

Chapter 3, Enhancing Applications, discusses a few different configurations that help

to enrich your data with Splunk knowledge objects, along with some basic App and
add-on branding guidelines.

Chapter 4, Basic Views and Dashboards, goes through the basics of SimpleXML
dashboard creation and development.

Chapter 5, The Splunk Web Framework, details the various SplunkJS Stack components
and shows examples of how to use them within an HTML dashboard.

Chapter 6, Advanced Integrations and Development, reviews modular inputs, data

models, the KV Store, and modular D3 visualizations.

Chapter 7, Packaging Applications, lists the items needed to package an App or add-on,
in order to get it ready for publishing.

Chapter 8, Publishing Applications, explains step by step how to upload an App to

Splunkbase, and includes some information on the great support community.

What you need for this book

To take full advantage of all the examples and code contained within this book, you
should have the following items:

• An installed and running instance of Splunk.

• Basic knowledge of how Splunk works, including searching, basic panels,
and dashboards.
• An understanding of the various technologies that Splunk uses. These
include the following:

°° Python
°° JavaScript
°° HTML
°° CSS

[ vii ]
Preface

Who this book is for

This book will benefit both the casual Splunker and the experienced professional
alike. Whether you are just starting Splunk Apps or add-on development, or have
been developing for years, this book has tips and tricks to help with developing new
integrations and Splunk Apps and add-ons. Even for quick modular input, this book
provides quick tutorials on common integration techniques and code examples.

Conventions
In this book, you will find a number of text styles that distinguish between different
kinds of information. Here are some examples of these styles and an explanation of
their meaning.

Code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles are shown as follows:
"Copy the file to $SPLUNK_HOME/etc/apps."

A block of code is set as follows:

[splunk_developers_guide]
coldPath = $SPLUNK_DB\splunk_developers_guide\colddb
homePath = $SPLUNK_DB\splunk_developers_guide\db
thawedPath = $SPLUNK_DB\splunk_developers_guide\thaweddb

Any command-line input or output is written as follows:

cd $APP_HOME/default

New terms and important words are shown in bold. Words that you see on the
screen, for example, in menus or dialog boxes, appear in the text like this: "Simply
click on the Browse button."

Warnings or important notes appear in a box like this.

Tips and tricks appear like this.

[ viii ]
 Preface

Reader feedback
Feedback from our readers is always welcome. Let us know what you think about
this book—what you liked or disliked. Reader feedback is important for us as it helps
us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail feedback@packtpub.com, and mention

the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing
or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support
Now that you are the proud owner of a Packt book, we have a number of things to
help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.
packtpub.com for all the Packt Publishing books you have purchased. If you
purchased this book elsewhere, you can visit http://www.packtpub.com/support
and register to have the files e-mailed directly to you.

Errata
Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you find a mistake in one of our books—maybe a mistake in the text or
the code—we would be grateful if you could report this to us. By doing so, you can
save other readers from frustration and help us improve subsequent versions of this
book. If you find any errata, please report them by visiting http://www.packtpub.
com/submit-errata, selecting your book, clicking on the Errata Submission Form
link, and entering the details of your errata. Once your errata are verified, your
submission will be accepted and the errata will be uploaded to our website or added
to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/

content/support and enter the name of the book in the search field. The required
information will appear under the Errata section.

[ ix ]
Preface

Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all
media. At Packt, we take the protection of our copyright and licenses very seriously.
If you come across any illegal copies of our works in any form on the Internet, please
provide us with the location address or website name immediately so that we can
pursue a remedy.

Please contact us at copyright@packtpub.com with a link to the suspected

pirated material.

We appreciate your help in protecting our authors and our ability to bring you
valuable content.

Questions
If you have a problem with any aspect of this book, you can contact us at
questions@packtpub.com, and we will do our best to address the problem.

[x]
 Application Design
Fundamentals
Hello there, Splunk developer! If you are like us, we know you have a love of Splunk
and all of the endless possibilities that present themselves! The Big Data world is
exploding around us, and it always feels like a tireless battle when keeping up to
date with advances in technologies, platforms, and concepts. Here, we will discuss
none of those. This book is dedicated solely to Splunk and the development of
applications for Splunk. Onward and upward!

What is a Splunk application?

All that being said, let's talk Splunk applications. A Splunk application is nothing
more than a structured set of configurations and assets used to achieve an end goal
of data collection, indexing, and visualization. Furthermore, in order to create a
valid Splunk application, you must include the ability to navigate. Without
navigation within the application, you would be working on an add-on.
According to Splunk, applications:

• Contain at least one navigable view

• Can be opened from the Splunk Enterprise home page, from the App menu,
or from the Apps section of Settings
• Focus on aspects of your data
• Are built around use cases
• Support diverse user groups and roles
• Run in tandem

[1]
Application Design Fundamentals

• Contain any number of configurations and knowledge objects

• Are completely customizable, from frontend to backend
• Can include Web assets such as HTML, CSS, and JavaScript

This is taken from http://docs.splunk.com/Documentation/

Splunk/latest/AdvancedDev/AppIntro.

Why applications?
Applications allow us to quickly share configurations, focus on the context of
available data, limit data access to specific individuals or groups, and organize
similar dashboards and views into a cohesive presentation of data within Splunk.
Sharing applications can be as easy as just zipping it up and sending it out. Splunk
applications could be said to be open source, due to the fact that almost all of the
configurations, custom scripts, and any other knowledge object contained within
the applications, are readable on the filesystem. This allows for customization for an
individual instance while maintaining an overall master configuration.

Definitions
To get started, we should define a few naming conventions typically used when
naming applications. Note that while we will use these naming conventions as
the best practice, your application can really be named anything at all, which
may conflict with other applications of the same name, or violate Splunk usage
agreements or publishing guidelines. In particular, the name Splunk cannot be
present in your application or add-on name. Additionally, in the past, Splunk has
referred to add-ons as technology add-ons, and has since moved to just add-ons.
The following list of add-on types is our way to distinguish the different uses
of each add-on:

• Applications: Applications could be named anything, as long as they are

relevant to the content of the application and don't contain the name Splunk.
• Domain add-ons (DA): Domain add-ons are not full applications, rather
they contain the visualizations and presentation of the data for a broader
application. No other configurations should be included (extracts, tags, event
types, macros, line breaking configurations, and so on). Dashboards and
views are the primary objects contained within this type of add-on.

[2]
 Chapter 1

• Supporting add-ons (SA): Supporting add-ons are also not full applications;
these contain data definitions, such as macros, saved searches, event types,
and tags. These describe how to correlate the data, normalize the data, and
consolidate the data to be usable in the domain add-on.
• Technology add-ons (TA): Technology add-ons provide extraction, data
massage, and index-time configurations. These can also be referred to as
technical add-ons. These contain the configuration options required to
properly break events, extract search fields, and create timestamps, among
other functions. These are the building blocks for the SA and DA add-ons,
as well as full-blown applications.

Follow the Splunk application design guidelines. Using a custom

naming scheme may cause conflicts.

Thus end the official naming conventions as normally seen in a Splunk installation.
We will now discuss some other naming conventions that have been found to help in
the wild west of various Splunk installations. These two naming conventions are of
the author's own design, which have helped in some of his deployments:

• Input add-ons (IA): Input add-ons are just that—configurations that assist
in the collection of data, known as inputs. These add-ons are most likely
found on a deployment server and are used to collect data from universal
forwarders. One of the advantages to splitting your IAs from your TAs is
a reduced size in the add-on being sent to the universal forwarder. This
is especially useful if your TA contains lookups that aren't needed on the
universal forwarder but are several megabytes in size.
• Admin add-ons (ADMIN): This add-on is a very special add-on. It would
typically contain administrative configurations that might be needed in a
variety of locations. Such configurations could be the web server SSL port,
deployment client information, or anything in web.conf or server.conf
format. It can be used to send index information to a set of non-clustered
indexers, or possibly to scale the addition of more search heads by setting
all relevant settings from a central location.

[3]
Application Design Fundamentals

While this may not be a complete list of naming conventions, it should be enough
to recognize any that are seen in the wild. An additional aspect of the naming
conventions that we recommend is the addition of company information. This will
help your Splunk admins differentiate between Splunk add-ons and custom add-ons.
Just as an example, let's say you built a TA for Cisco, specific to your company (the
ACME company). Splunk's provided add-on is entitled TA-cisco, but you don't want
to modify a vendor's offering. So, your new add-on's name could be A-ACME-TA-
cisco. This gives you two things: an easy-to-see custom TA that relates to Cisco and
the ability to override any TA-cisco settings based on application precedence.

Let's discuss application precedence for a moment. Splunk uses a merged configuration
when applying configurations that are installed via the applications. The
methodology that Splunk chose to implement conflict resolution is pretty simple.
There are two different methods of precedence. The first is directory structure. If you
have an input located in the default folder of an application (more on default in
the later chapters), you can place a matching configuration in the local folder of the
application to override the default configuration. The same method is applied to
the applications themselves. Splunk uses the ASCII values of the names to determine
precedence. On *nix, you can sort the applications in the apps folder of Splunk using
the LC_COLLATE=C ls command. This will show you the ASCII-sorted order of the
applications, and the first in the list will be highest priority. A has a higher priority
than Z, but Z has a higher priority than a. So, the A at the beginning of the add-on
name gives your add-on the highest precedence, so you can override any setting
as needed.

From this point forward, both Splunk applications and add-ons

will be referred to formally as Apps purely as a convenience.

Designing the App

So you've decided that you need an App? Congratulations! Now that you know that
you need one, you need to decide on a few more items as well. It is important to do
a little bit of planning, as even the simplest Apps can evolve into super-complicated
Apps, with dashboards, saved searches, workflows, and more. Never assume "well,
this'll just be a quick development", as, most of the time, it is not.

[4]
 Chapter 1

Identifying the use case

First and foremost, try to determine the scope of your App. Once you have the scope
planned out, try to limit the amount of scope creep that occurs, if possible. You may
just be trying to perform extractions on your data, and if that is your current end
goal, stop there. Don't try to build a full-blown suite on your first attempt. Build the
IA, then the TA, and then move on from there. Ask yourself these questions as you
try to determine your scope:

• What am I trying to accomplish? Search-time extractions? Index-time

parsing? Dashboards to share?
• What users need access to my App? Everybody? Specific roles?
• What kind of information will I be presenting? Server based? Metric based?
• Who is my target audience? Business users who don't understand Splunk
Search Processing Language (SPL), or technical users who will notice that I
didn't convert MB to GB properly?

These questions can help you spark an idea of what internal resources would need to
be engaged, as well as any kind of documentation and educational requirements.

Identifying what you want to consume

Once you have determined the scope of the App, you will need to decide how and
from where you will consume the data. Getting data into Splunk can happen in a
very wide variety of ways. There is no set manner of input that will work on all data
sources. You may have to develop a new script or modular input. Being aware of
where your data is coming from is the key to getting it consumed correctly the first
time. A few questions you may ask yourself could be:

• Why do I need this data? Is it all completely relevant to my use case?

• Where is the data? Cloud, SaaS provider, internal network?
• How do I get the data? Do I already have a collector script, or do I need to
engage an internal resource to write a collector/modular input?
• What format is the data? Is it already extracted (or well known, like syslog),
or do I need to write custom extractions?

There is a lot of data out in the wild, but not all of it may be relevant to your use case.
You may find that of a service that has 100 endpoints available for data collection,
you only need 10. Not only will you save on license usage, but your indexers will
thank you for it as well.

[5]
Application Design Fundamentals

Identifying what you want to brand

Another key thought process in App development is how far you want to brand your
App. Splunk has a very robust architecture and framework, providing you with the
ability to customize your Apps extensively. You can override any individual piece of
CSS and extend SplunkJS Stack to include any number of different visualizations or
third-party libraries. Additional questions you might ponder on would include:

• Do I want to brand anything at all, or just stay with native Splunk?

• Do I need to engage an internal graphics resource to design and create App
icons? App logos?
• Am I going for mobile or static desktops? What desktop size is typical of
incoming users?
• To what extent should I customize my App? Do I just change a few colors
using native Splunk options or do I override CSS?
• Do I need to engage a web designer to build custom CSS or HTML layouts?

There are so many options available to brand your App, but all customizations
should conform to the Splunk branding guidelines for developers. Visit http://
www.splunk.com/view/SP-CAAAFT9 to read through Splunk's guidelines.

Identifying what you want to display

Once you have the whats and hows of the data you're going to collect, you need
to figure out visualizations. How you display the information is just as important
as what data you collect. Splunk comes with a variety of graphs and displays
right out of the box, and can be extended quite easily to include some really cool
presentations. Some of the questions posed to you might be:

• Do you need a programmer to write custom modules or extend SplunkJS

views and managers?
• What third-party graphing or graphic libraries do you need to document,
develop, or get permission to use?
• Do you need to engage a statistician to determine the best and most effective
way to display your data? Some stats (such as max, mean, and min) are easy,
others (such as confidence intervals and trendlines) are not.

[6]
 Chapter 1

Such a small list of questions hardly precludes any other relevant discussion within
your organization. The more internal discussion that can take place, the better and
more thought-out your App may turn out.

Installing Apps
As a Splunk developer, you should be aware of the three methods to install Apps.
There are advantages and disadvantages to each method, but no required method.
It is mostly personal preference as to which method is used by the end user, but,
typically, newer Splunk users will use the Web interface, while advanced users
will use the command line. Let's review those methods, just to keep them fresh in
your mind.

Splunk Web
Installing Apps via Splunk Web is simple. Once you have downloaded the App from
its source, you navigate to the Manage Apps section of Splunk. You will find this at
the top-left of Splunk Web, as shown in the following screenshot:

Once you have clicked on Manage Apps, you will see a button to install the app
from a file. You can also browse the Splunk App store, using the first button:

[7]
Application Design Fundamentals

This brings you to a form that you can use to actually install the App. Simply click on
the Browse button, select the file you downloaded, check the Upgrade button if this
App has already been installed, and then click on Upload. That's it! Splunk takes the
App, installs it, and prompts to restart if needed:

The Splunk command line

CLI holds a special place in many *nix admins' hearts. It is entirely possible to install
Apps via the command line alone. Doing so requires having the following: access to
the physical (or virtual) server and enough permissions to perform CLI commands
with Splunk. All commands are going to be executed from $SPLUNK_HOME, which
normally defaults to /opt/splunk. Follow these steps to install an App via CLI:

1. Copy the App file (either a *.tgz or *.spl file) to the filesystem.
2. Run the ./bin/splunk install app <path_to_file> command.

Splunk will install the App. You may be prompted to restart, depending on the
contents of the App. Index-time configurations require a restart, whereas search-time
configurations do not.

Unzipping using the command line

The final methodology is to perform an unzip/untar. If the App was constructed
properly, the only steps you need to perform are as follows:

1. Copy the file to $SPLUNK_HOME/etc/apps.

2. Change the file extension from .spl to .zip.
3. Use your favorite utility and unzip the file into the folder.

[8]
 Chapter 1

Caution! This will overwrite any other settings you have

configured, including local configurations (if present in the zip
file). We will cover directory structure in the next chapter.

Downloading the example code

You can download the example code files from your account at
http://www.packtpub.com for all the Packt Publishing books
you have purchased. If you purchased this book elsewhere, you can
visit http://www.packtpub.com/support and register to have
the files e-mailed directly to you.

Summary
In this chapter, we covered the basic fundamentals of designing and installing
Splunk Apps. Apps can be broken down into domains, each with a naming
convention that allows you to quickly determine what the App can do, and what is
contained within it, so that new users to your environment don't have to look for
configurations. We learned how to approach App design to make sure that the App
is planned beforehand, which will eliminate the need to refactor major portions of
the App later, when it may already be in production. We also went over the three
different methodologies available to install Apps to give a basic understanding of
user experience related to the installation of any App you may build.

Now that you've acquired an understanding of what an App consists of, in the
coming chapters, we will discuss creating, enhancing, and customizing them.

[9]
 Creating Applications
In this chapter, we will begin covering how to build an actual application. There are
many different ways to create an App, ranging from GUI creation to manual editing
of configuration files. We will cover the structure of an application, what each folder
should contain within the application, and why this is important. Another aspect
that will be touched on will be the data that your application will consume. Setting
up the data structures beforehand may save you time and energy later on if you have
to refactor. It is crucial to get the data in correctly the first time, as any subsequent
release of your app will need to make use of the data. We will cover various methods
for data consumption, as well as the types of Splunk knowledge objects that can be
included in your application. Restricting access to your application may be a priority,
so we will also cover metadata and object permissions. Getting your application
installed may require your end user to perform some additional configuration before
it can be used, so we will review how to configure the setup screen as well.

A brief clarification
As we continue to progress through this book, we will create an App from the
ground up. The App's name is SDG (from a filesystem perspective) and the App label
will be Developer's Guide for Splunk. It will be available in its entirety on Splunkbase
at https://splunkbase.splunk.com/app/2693/. Additionally, we will be using
an API provided by meh.com, a daily deal site that was kind enough to build an API
to their website. They were chosen primarily because they fit the geek culture pretty
well, and provide a very simple-to-consume API. The data that will be consumed is
pulled from their website's API using scripted inputs located in the bin folder of the
sample SDG application.

[ 11 ]
Creating Applications

Let's recall the questions from Chapter 1, Application Design Fundamentals, that revolve
around App creation. We should answer some of them in preparation for building
our demo App:

• Identifying the use case:

°° We are building this App as a learning experience for the reader. By
providing an App at the end of this book, with all the examples from
the book contained within the App, we will give you a means to see a
final product, as well as how it was created, step by step.
°° We are building an App with visualizations and modular inputs that
will be shared with everybody—no need for role-based access.
°° The data will primarily consist of daily deals information, along with
polling information.

• Identifying what you want to consume:

°° The data is needed as an event generator for our App building demo,
and is located at an API provided by meh.com. I will consume the
data with a modular input from the API, which outputs its data
in JSON.

• Identifying what you want to brand:

°° For demonstration purposes, we will brand the App with custom
icons and some custom CSS and JavaScript. No external resources
will be needed.

• Identifying what you want to display:

°° We will be using a box plot graphic, which needs to be transformed

into a modular Splunk JavaScript library. The statistics will be
simplistic, so a mathematician is not required.

Now that we have answered some preliminary questions, we are ready to begin
creating our App. We can clearly see what is needed at the very basic level, and can
continue to add to the specification as we go ahead.

Methods of creating applications

There are two basic ways of creating applications. They are as follows, in order of
difficulty (not that any of them are hard): Splunk Web (we will call this the GUI),
and handwritten (henceforth to be recognized as FreeForm). In order to
create Apps, you, the developer, must have specific permissions within the
Splunk instance.

[ 12 ]
Exploring the Variety of Random
Documents with Different Content
 "And you told him about Clare's father, I suppose?"
"No," she answered. "Don't be angry," she pleaded, laying a hand
on his arm. "I don't know what made me do it—I suppose it was
instinct. Anyway, you were going to, soon, even if I hadn't. I—I told
father about—us!"
"You did?"
"Yes. Don't be angry with me."
"My darling, I'm not angry with you. What did he say?"
She came so close to him that he could feel her body trembling
with emotion. "He didn't mind," she whispered. "He didn't mind at all.
Kenneth, aren't you glad?—Isn't it fine of him?"
"Glorious!" he answered, taking a deep breath. Again the tide of
joy seemed to engulf him, joy immense and stupefying. He would
have taken her in his arms and kissed her had he not seen people
coming along the lane. "It's wonderful, Helen!" he whispered. Then
some secondary thought seemed to strike him suddenly: he said:
"But why were you miserable a little while ago? Didn't the good news
make you feel happy?"
She answered, still with a touch of sadness: "I didn't know
whether you would think it was good news."—"Helen!" he exclaimed,
remonstratively, clasping her tightly to him: she went on, smiling at
him: "Yes, it's silly of me, isn't it?—But Kenneth, Kenneth, I don't
know how it is, I'm never quite certain of you—there's always a funny
sort of fear in my mind! I know it's silly. I can't help it, though.
Perhaps it will all be different some day."
"Some day!" he echoed, gazing into her uplifted eyes.
A vision, secret and excruciatingly lovely, filled their eyes for a
moment. He knew then that to marry her had become his blinding
and passionate ambition.
 V

The Millstead and District Advertiser had a long and sympathetic

appreciation of the late Mr. Samuel Harrington in its first July issue.
The Helping-Hand-Books were described as "pleasant little homilies
written with much charm and humour." Speed took one or two of
them out of the School Library and read them.
About a week after the funeral he called at the shop, ostensibly to
buy a book, but really to offer his condolences. He had been
meaning to go, for several days in succession, but a curious dread of
an interview with Clare had operated each time for postponement.
Nor could he understand this dread. He tried to analyse it, to
discover behind it any conceivable reason or motive; but the search
was in vain. He was forced to suppose, vaguely, that the cause of it
was that slight but noticeable temperamental hostility between
himself and Clare which always resulted in a clouding over of his
dreams.
It was a chilly day for July; there was no sun, and the gas was
actually lit in the shop when he called. The boy, a smart under-sized
youngster, was there to serve him, but he asked for Miss Harrington.
She must have heard his voice, for she appeared almost
straightway, dressed neatly and soberly in black, and greeted him
with a quite brisk: "Good afternoon, Mr. Speed!"
He shook hands with her gravely and began to stammer: "I should
have called before, Miss Harrington, to offer you my sincerest
sympathies, but—"
She held up her hand in an odd little gesture of reproof and said
interrupting him: "Please don't. If you want a chat come into the back
room. Thomas can attend to the shop."
He accepted her invitation almost mechanically. It was a small
room, full of businesslike litter such as is usual in the back rooms of
shops, but a piano and bookcase gave it a touch of individuality. As
she pointed him to a seat she said: "Don't think me rude, but this is
the place for conversation. The shop is for buying things. You'll know
in future, won't you?"
 He nodded somewhat vaguely. He could not determine what
exactly was astounding in her, and yet he realised that the whole
effect of her was somehow astounding. More than ever was he
conscious of the subtle hostility, by no means amounting to
unfriendliness, but perhaps importing into her regard for him a tinge
of contempt.
"Do you know," he said, approaching the subject very deliberately,
"that until a very short time ago I knew nothing at all about Mr.
Harrington? You never told me."
"Why should I?" She was on her guard in an instant.
He went on: "You may think me sincere or not as you choose, but
I should like to have met him."
"He had a dislike of being met."
She said that with a touch of almost vicious asperity.
He went on, far less daunted by her rudeness than he would have
been if she had given way to emotion of any kind: "Anyway I have
got to know him as well as I can by reading his books."
"What a way to get to know him!" she exclaimed, contemptuously.
She looked him sternly in the face and said: "Be frank, Mr. Speed,
and admit that you found my father's books the most infantile trash
you ever read in your life!"
"Miss Harrington!" he exclaimed, protesting. She rose, stood over
him menacingly, and cried: "You have your chance to be frank,
mind!"
He looked at her, tried to frame some polite reply, and found
himself saying astonishingly: "Well, to be perfectly candid, that was
rather my opinion."
"And mine," she added quietly.
She was calm in an instant. She looked at him almost
sympathetically for a moment, and with a sudden gesture of
satisfaction sat down in a chair opposite to his. "I'm glad you were
frank with me, Mr. Speed," she said. "I can talk to anybody who's
frank with me. It's your nature to confide in anybody who gives you
the least encouragement, but it's not mine. I'm rather reticent. I
remember once you talked to me a lot about your own people.
Perhaps you thought it strange of me not to reciprocate."
"No, I never thought of it then."
"You didn't?—Well, I thought perhaps you might have done. Now
that you've shown yourself candid I can tell you very briefly the sort
of man my father was. He was a very dear old hypocrite, and I was
very fond of him. He didn't feel half the things he said in his books,
though I think he was honest enough to try to. He found a good thing
and he stuck to it. After all, writing books was only his trade, and a
man oughtn't to be judged entirely by what he's forced to do in order
to make a living."
He stared at her half-incredulously. She was astounding him more
than ever. She went on, with a curious smile: "He was fifty-seven
years old. When he died he was half-way through his eleventh book.
It was to have been called 'How to Live to Three-Score-Years-and-
Ten.' All about eating nuts and keeping the bedroom windows open
at nights, you know."
He wondered if he were expected to laugh.
He stammered, after a bewildered pause: "How is all this going to
effect you?—Will you leave Millstead?"
She replied, with a touch in her voice of what he thought might
have been mockery: "My father foresaw the plight I might be in some
day and thoughtfully left me his counsel on the subject. Perhaps
you'd like me to read it?"
She went over to the bookcase and took down an edition-de-luxe
copy of one of the Helping-Hand-Books.
"Here it is—'How to Meet Difficulties'—Page 38—I'll read the
passage—it's only a short one. 'How is it that the greatest and
noblest of men and women are those against whom Fate has set her
most tremendous obstacles?—Simply that it is good for a man or a
woman to fight, good to find paths fraught with dire perils and
difficulties galore, good to accept the ringing challenge of the gods!
Nay, I would almost go so far as to say: lucky is that boy or girl who
is cast, forlorn and parentless upon the world at a tender age, for if
there be greatness in him or her at all, it will be forced to show itself
as surely as the warm suns of May compel each flower to put forth
her bravest splendour!' ... So now you know, Mr. Speed!"
She had read the passage as if declaiming to an audience. It was
quite a typical extract from the works of the late Mr. Harrington: such
phrases as 'dire perils,' 'difficulties galore,' and 'ringing challenge of
the gods' contained all that was most truly characteristic of the prose
style of the Helping-Hand-Books.
Speed said, rather coldly: "Do you know what one would wonder,
hearing you talk like this?"
"What?"
"One would wonder if you had any heart at all."
Again the curious look came into her eyes and the note of asperity
into her voice. "If I had, do you think I would let you see it, Mr.
Speed?" she said.
They stared at each other almost defiantly for a moment; then, as
if by mutual consent, allowed the conversation to wander into
unimportant gossip about Millstead. Nor from those placid channels
did it afterwards stray away. Hostility of a kind persisted between
them more patently than ever; yet, in a curiously instinctive way, they
shook hands when they separated as if they were staunch friends.
As he stepped out into High Street the thought of Helen came to
him as a shaft of sunlight round the edges of a dark cloud.

VI

Term finished in a scurry of House-matches and examinations.

School House won the cricket trophy and there was a celebratory
dinner at which Speed accompanied songs and made a nervously
witty speech and was vociferously applauded. "We all know we're
the best House," said Clanwell, emphatically, "and what we've got to
do is just to prove to other people that we are." Speed said: "I've only
been in School House a term, but it's been long enough time for me
to be glad I'm where I am and not in any other House." (Cheers.)
Amidst such jingoist insincerities a very pleasant evening romped its
way to a close. The following day, the last day of term, was nearly as
full of new experiences as had been the first day. School House yard
was full of boxes and trunks waiting to be collected by the railway
carriers, and in amongst it all, small boys wandered forlornly, secretly
happy yet weak with the cumulative passion of anticipation. In the
evening there was the farewell dinner in the dining-hall, the
distribution of the terminal magazine, and the end-of-term concert,
this last concluding with the Millstead School-Song, the work of an
uninspired composer in one of his most uninspired moments. Then,
towards ten o'clock in the evening, a short service in chapel,
followed by a "rag" on the school quadrangle, brought the long last
day to a close. Cheers were shouted for the Masters, for Doctor and
Mrs. Ervine, for those leaving, and (facetiously) for the school porter.
That night there was singing and rowdyism in the dormitories, but
Speed did not interfere.
He was ecstatically happy. His first term had been a triumph. And,
fittingly enough, it had ended with the greatest triumph of all. Ever
since Helen had told him of her confession to her father, Speed had
been making up his mind to visit the Head and formally put the
matter before him. That night, the last night of the summer term,
after the service in chapel, when the term, so far as the Head was
concerned with it, was finished. Speed had tapped at the door of the
Head's study.
Once again the sight of that study, yellowly luminous in the
incandescent glow, set up in him a sensation of sinister attraction, as
if the room were full of melancholy ghosts. The Head was still in his
surplice, swirling his arms about the writing-table in an endeavour to
find some mislaid paper. The rows upon rows of shining leather-
bound volumes, somebody on the Synoptic Gospels, somebody
else's New Testament Commentary, seemed to surround him and
enfold him like a protective rampart. The cool air of the summer night
floated in through the slit of open window and blew the gas-light
fitfully high and low. Speed thought, as he entered the room and saw
the Head's shining bald head bowed over the writing-table: Here you
have been for goodness knows how many years and terms, and now
has come the end of another one. Don't you feel any emotion in it at
all?—You are getting to be an old man: can you bear to think of the
day you first entered this old room and placed those books on the
shelves instead of those that belonged to your predecessor?—Can
you bear to think of all the generations that have passed by, all the
boys, now men, who have stared at you inside this very room, while
time, which bore them away in a happy tide, has left you for ever
stranded?—Why I, even I, can feel, after the first term, something of
that poignant melancholy which, if I were in your place, would
overwhelm me. Don't you—can't you—feel anything at all—
The Head looked up, observed Speed, and said: "Um, yes—
pleased to see you, Mr. Speed—have you come to say good-bye—
catching an early train to-morrow, perhaps—um, yes—eh?"
"No, sir. I wanted to speak to you on a private matter. Can you
spare me a few moments?"
"Oh yes, most certainly. Not perhaps the—um—usual time for
seeing me, but still—that is no matter. I shall be—um—happy to talk
with you, Mr. Speed."
Speed cleared his throat, shifted from one foot to another, and
began, rather loudly, as always when he was nervous: "Miss Ervine,
sir, I believe, spoke to you some while ago about—about herself and
me, sir."
The Head placed the tips of his fingers together and leaned back
in his chair.
"That is so, Mr. Speed."
"I—I have been meaning to come and see you about it for some
time. I hope—I hope you didn't think there was anything underhand
in my not seeing you?"
 The Head temporised suavely: "Well—um, yes—perhaps my
curiosity did not go so—um—so far as that. When you return to your
room, Mr. Speed, you will find there an—um—a note from me,
requesting you to see me to-morrow morning. I take it you have not
seen that note?"
"Not yet, sir."
"Ah, I see. I supposed when you entered that you were catching
an early train in the morning and were—um—purposing to see me
to-night instead.... No matter. You will understand why I wished to
see you, no doubt."
"Possibly the same reason that I wished to see you."
"Ah, yes—possibly. Possibly. You have been—um—quite—um—
speedy—in—um— pressing forward your suit with my daughter. Um,
yes—very speedy, I think.... Speedy—Ha—Ha—um, yes—the play
upon words was quite accidental, I assure you."
Speed, with a wan smile, declared: "I daresay I am to blame for
not having mentioned it to you before now. I decided—I scarcely
know why—to wait until term was over.... I—I love your daughter,
and I believe she loves me. That's all there is to say, I think."
"Indeed, Mr. Speed?—It must be a very—um—simple matter
then."
Speed laughed, recovering his assurance now that he had made
his principal statement. "I am aware that there are complexities, sir."
The Head played an imaginary tune on his desk with his
outstretched fingers. "You must—u—listen to me for a little while, Mr.
Speed. We like you very much—I will begin, perhaps unwisely, by
telling you that. You have been all that we could have desired during
this last term—given—um—every satisfaction, indeed. Naturally, I
think too of my daughter's feelings. She is, as you say, extremely—
um—fond of you, and on you depends to a quite considerable extent
her—um—happiness. We could not therefore, my wife and I, refuse
to give the matter our very careful consideration. Now I must—um—
cross-examine you a little. You wish to marry my daughter, is that not
so?"
 "Yes."
"When?"
The Head flung out the question with disconcerting suddenness.
Speed, momentarily unbalanced, paused, recovered himself, and
said wisely: "When I can afford to, sir. As soon as I can afford to. You
know my salary and prospects, sir, and are the best judge of how
soon I shall be able to give your daughter the comforts to which she
has been accustomed."
"A clever reply, Mr. Speed. Um, yes—extremely clever. I gather
that you are quite convinced that you will be happy with my
daughter?"
"I am quite convinced, sir."
"Then money is the only difficulty. What a troublesome thing
money is, Mr. Speed!—May I ask you whether you have yet
consulted your own parents on the matter?"
"I have not done so yet. I wanted your reply first."
"I see. And what—um—do you anticipate will be their reply?"
Speed was silent for a moment and then said: "I cannot pretend
that I think they will be enthusiastic. They have never agreed with my
actions. But they have the sense to realise that I am old enough to
do as I choose, especially in such a matter as marriage. They
certainly wouldn't quarrel with me over it."
The Head stared fixedly at Speed for some while; then, with a
soft, crooning tone, began to speak. "Well, you know, Mr. Speed, you
are very young—only twenty-two, I believe."—(Speed interjected:
"Twenty-three next month, sir.")—The Head proceeded: "Twenty-
three then. It's—um—it's rather young for marriage. However, I am—
um, yes—inclined to agree with Professor Potts that one of the—um
—curses of our modern civilisation is that it pushes the—um—
marriageable age too late for the educated man." (And who the devil,
thought Speed, is Professor Potts?)... "Now it so happens, Mr.
Speed, that this little problem of ours can be settled in a way which is
satisfactory to myself and to the school, and which I think will be
equally satisfactory to yourself and my daughter. I don't know
whether you know that Lavery leaves this term?"
"I didn't know, sir."
"He has reached the—um—the retiring age. As perhaps you
know, Mr. Speed, Lavery belonged to the—um—old school. In many
ways, I think, the old methods were best, but, of course, one has to
keep up with the times. I am quite certain that the Governors will look
favourably on a very much younger man to be—um—Lavery's
successor. It would also bean advantage if he were married."
"Married!" echoed Speed.
"Yes. Married house masters are always preferred.... Then, again,
Mr. Speed, we should want a public-school man.... Of course,
Lavery's is a large House and the position is not one to be—um—
lightly undertaken. And, of course, it is for the Governors to decide,
in the last resort. But if you think about it, Mr. Speed, and if you
favour the idea, it will probably occur to you that you stand a rather
good chance. Of course it requires thinking over a great deal. Um,
yes—decide nothing in a hurry...."
Speed's mind, hazily receiving the gist of what the Head was
saying, began to execute a wild pirouette. He heard the Head's voice
droning on, but he did not properly hear anything more that was said.
He heard in snatches: "Of course you would have to take up your
new duties in—um, yes—September.... And for that purpose you
would get married during the vacation.... A great chance for you, Mr.
Speed ... the Governors ... very greatly impressed with you at
Speech Day.... You would like Lavery's .... an excellent House....
Plenty of time to think it over, you know.... Um, yes—plenty of time....
When did you say you were going home?"
Speed recovered himself so far as to answer: "Tuesday, sir."
"Um, yes—delightful, that is—you will be able to dine with us to-
morrow night then, no doubt?—Curious place, Millstead, when
everybody has gone away... Um, yes—extremely delightful... Think it
over very carefully, Mr. Speed ... we dine at seven-thirty during the
vacations, remember.... Good night, then, Mr. Speed.... Um, yes—
Good night!"
Speed staggered out as if intoxicated.

VII

That was why, hearing the singing and shouting in the dormitories
that night, Speed did not interfere. With happiness surging all around
him how could he have the heart to curtail the happiness of others?
—About half-past ten he went round distributing journey-money, and
to each dormitory in turn he said farewell and wished a pleasant
vacation. The juniors were scampering over one another's beds and
pelting one another with pillows. Speed said merely: "If I were you
fellows, I should get to sleep pretty soon: Hartopp will ring the bells
at six, you know."
Then he went back into his own room, his room that would not be
his any more, for next term he would be in Lavery's. Noisy and
insincere as had been his protestations at the House Dinner about
the superiority of School House over any other, there was yet a
sense in which he felt deeply sorry to leave the place where he had
been so happy and successful. He looked back in memory to that
first evening of term, and remembered his first impression of the
room assigned to him; then it had seemed to him lonely, forlorn,
even a little dingy. Hardly a trace of that earliest aspect remained
with it now. At eleven o'clock on the last night of term it glowed with
the warmth of a friendly heart; it held out loving arms that made
Speed, even amidst his joy, piteously sorry to leave it. The empty
firegrate, in which he had never seen a fire, lured him with the vision
of all the cosy winter nights that he had missed.
Outside it was moonlight again, as when, a month before, he had
waited by the pavilion steps on the evening of the Speech Day. From
his open lattice-window he could see the silver tide lapping against
the walls and trees, the pale sea of the pitch on which there would
be no more cricket, the roof and turret of the pavilion gleaming with
liquid radiance. All was soft and silent, glossy beneath the high
moon. It was as if everything had endured agelessly, as if the
passing of a term were no more than the half-heard tick of a clock in
the life of Millstead.
Leaning out of the window he heard a voice, boyish and sudden,
in the junior dormitory below.
"I say, Bennett, are you going by the 8:22?"
An answer came indistinguishably, and then the curt command of
the prefect imposing silence, silence which, reigned over by the
moon and the sky of stars, lasted through the short summer night
until dawn.

BOOK II

THE WINTER TERM

CHAPTER ONE

He breakfasted with Helen upon the first morning of the winter

term, inaptly named because winter does not begin until the term is
over. They had returned the evening before from a month's holiday
in Cornwall and now they were making themselves a little self-
consciously at home in the first-floor rooms that had been assigned
to them in Lavery's. The room in which they breakfasted overlooked
the main quadrangle; the silver coffee-pot on the table shimmered in
the rays of the late summer sun.
Midway through the meal Burton, the porter at Lavery's, tapped at
the door and brought in the letters and the Daily Telegraph.
Speed said: "Hullo, that's luck!—I was thinking I should have to
run down the town to get my paper this morning."
Burton replied, with a hint of reproach in his voice: "No sir. It was
sent up from Harrington's as usual, sir. They always begin on the
first day of term, sir."
Speed nodded, curiously conscious of a thrill at the mention of the
name Harrington. Something made him suddenly nervous, so that he
said, boisterously, as if determined to show Burton at all costs that
he was not afraid of him: "Oh, by the way, Burton, you might shut
that window a little, will you?—there's a draught."
"Yes, sir," replied Burton, again with a hint of subdued
reproachfulness in his tone. While he was shutting the window,
moving about very softly and stealthily himself yet making a
tremendous noise with everything that had to be done with his large
clumsy hands, it was necessary for Speed and Helen to converse on
casual and ordinary subjects.
Speed said: "I should think the ground's far too hard for rugger,
Helen."
She answered, somberly: "Yes, I daresay it is. It's really summer
still, isn't it?—And I'm so glad. I hate the winters."
"You hate the winters, eh?—Why's that?"
"It's so cold, and the pitches are all muddy, and there are horrid
locks-up after dark. Oh, I hate Millstead in winter-time."
He said, musingly: "We must have big fires when the cold weather
comes, anyway."
Then Burton departed, closing the door very delicately after him,
and the conversation languished. Speed glanced vaguely through
his correspondence. He was ever so slightly nervous. That month's
honeymoon in Cornwall had passed like a rapt and cloudless vision,
but ever beyond the horizon of it had been the thought of this return
to Millstead for the winter term. How the return to a place where he
had been so happy and of which he had such wonderful memories
could have taken in his mind the semblance of an ordeal, was a
question that baffled him entirely. He felt strangely and
unaccountably shy of entering the Masters' Common-Room again, of
meeting Clanwell and Ransome and Pritchard and the rest, of seeing
once again all the well-known faces of the boys whose summer
vacations had been spent so much less eventfully than his. And yet,
as he sat at the breakfast-table and saw Helen opposite him, a
strange warming happiness surged up within him and made him long
for the initial ordeal to be over so that he could pass on to the pure
and wonderful life ahead, that life in which Helen and Millstead would
reign jointly and magnificently. Surely he could call himself blessed
with the most amazing good fortune, to be happily married at the age
of twenty-three and installed in just the position which, more than
any other in the world, he had always coveted!—Consciousness of
his supreme happiness made him quicken with the richest and most
rapturous enthusiasm; he would, he decided in a sudden blinding
moment, make Lavery's the finest of all the houses at Millstead; he
would develop alike the work and the games and the moral tone until
the fame of Lavery's spread far beyond its local boundaries and
actually enhanced the reputation of Millstead itself. Such
achievements were not, he knew, beyond the possibilities of an
energetic housemaster, and he, young and full of enthusiasms,
would be a living fount of energy. All the proud glory of life was
before him, and in the fullness of that life there was nothing that he
might not do if he chose.
All that day he was at the mercy alternately of his tumultuous
dreams of the future and of a presaging nervousness of the
imminent ordeals. In the morning he was occupied chiefly with
clerical work, but in the afternoon, pleading a few errands in town, he
took his bicycle out of the shed and promised Helen that he would
not be gone longer than an hour or so. He felt a little sad to leave
her, because he knew that with the very least encouragement she
would have offered to come with him. Somehow, he would not have
been pleased for her to do that; he felt acutely self-conscious,
vaguely yet miserably apprehensive of trials that were in wait for him.
In a few days, of course, everything would be all right.
He did not cycle into the town, but along the winding Millstead
lane that led away from the houses and into the uplands around
Parminters. The sun was glorious and warm, and the trees of that
deep and heavy green that had not, so far, more than the faintest of
autumnal tints in it. Along the lane twisting into the cleft of
Parminters, memories assailed him at every yard. He had been so
happy here—and here—and here; here he had laughed loudly at
something she had said; here she had made one of her childish yet
incomparably wise remarks. Those old serene days, those splendid
flaming afternoons of the summer term, had been so sweet and
exquisite and fragrantly memorable to him that he could not forbear
to wonder if anything could ever be so lovely again.
Deep down beneath all the self-consciousness and apprehension
and morbid researching of the past, he knew that he was richly and
abundantly happy. He knew that she was still a child in his own mind;
that life with her was dream-like as had been the rapt anticipation of
it; that the dream, so far from marriage dispelling it, was enhanced
by all the kindling intimacies that swung them both, as it were, into
the same ethereal orbit. When he thought about it he came to the
conclusion that their marriage must be the most wonderful marriage
in the world. She was a child, a strange, winsome fairy-child,
streaked with the most fitful and sombre passion that he had ever
known. Nobody, perhaps, would guess that he and she were man
and wife. The thought gratified his sense of the singularity of what
had happened. At the Cornish hotel where they had stayed he had
fancied that their identity as a honeymoon couple had not been
guessed at all; he had thought, with many an inward chuckle, that
people were supposing them to be mysteriously and romantically
unwedded.
Time passed so slowly on that first day of the winter term. He rode
back at the end of the afternoon with the wind behind him, swinging
him in through the main gateway where he could see the windows of
Lavery's pink in the rays of the setting sun. Lavery's!—Lavery's!—
Throughout the day he had found himself repeating the name
constantly, until the syllables lost all shred of meaning. Lavery's!—
Lav-er-izz.... The sounds boomed in his ears as he entered the tiny
drawing-room in which Helen was waiting for him with the tea almost
ready. Tea time!—In a few hours the great machine of Millstead
would have begun to pound its inexorable way. He felt, listening to
the chiming of the quarters, as if he were standing in the engine-
room of an ocean liner, watching the mighty shafts, now silent and
motionless, that would so soon begin their solemn crashing
movement.
But that evening, about eleven o'clock, all his fears and
shynesses were over, and he felt the most deeply contented man in
the world. A fire was flickering a cheerful glow over the tiny drawing-
room; Helen had complained of chilliness so he had told Burton to
light it. He was glad now that this had been done, for it enabled him
to grapple with his dreams more comfortably. Helen sat in an
armchair opposite to him on the other side of the fire; she was
leaning forward with her head on her hands, so that the firelight
shone wonderfully on her hair. He looked at her from time to time,
magnificently in love with her, and always amazed that she belonged
to him.
The long day of ordeals had passed by. He had dined that
evening in the Masters' Common-Room, and everybody there had
pressed round him in a chorus of eager congratulation. Afterwards
he had toured his House, introducing himself to those of the prefects
whom he had not already met, and strolling round the dormitories to
shake hands informally with the rank and file. Then he had
interviewed new boys (there were nineteen of them), and had
distributed a few words of pastoral advice, concluding with the strict
injunction that if they made tea in the basements they were on no
account to throw the slops down the waste-pipes of the baths. Lastly
of all, he had put his head inside each of the dormitories, at about
half-past ten, to bestow a brisk but genial good-night.
So now, at eleven o'clock, rooted at last in everything that he most
loved in the world, he could pause to gloat over his happiness. Here,
in the snug firelit room, secret and rich with warm shadows; and
there, down the short corridor into the bleak emptiness of the
classrooms, was everything that his heart desired: Helen and
Millstead: the two deities that held passionate sway over him.—
Eleven began to chime on the school clock. The dormitories above
were almost silent. She did not speak, did not look up once from the
redness of the fire; she was often like that, silent with thoughts
whose nature he could guess from the dark, tossing passion that
shook her sometimes when, in the midst of such silences, she
suddenly clung to him. She loved him more than ever he could have
imagined: that, more perhaps than anything else, had been the
surprise of that month in Cornwall.
"Eleven," he said, breaking the rapt silence.
She said, half humorously, half sadly: "Are you pleased with me?
—Are you satisfied?—Do I quite come up to expectations?"
He started, looked towards her, and laughed. The laugh disturbed
the silence of the room like the intrusion of something from millions
of miles away. He made a humorous pretence of puzzling it out, as if
it were a baffling problem, and said, finally, with mocking
doubtfulness: "Well, on the whole, I think you do."
"If I had been on trial for a month you'd still keep me, then?" she
went on, without moving her head out of her hands.
He answered, in the same vein as before: "If you could guarantee
always to remain up to sample, I daresay I would."
She raised her head and gave him such a look as, if he had not
learned to know it, would have made him think she was angry with
him; it was sharp with blade-like eagerness, as if she were piercing
through his attitude of jocularity.
Then, wondering why she did not smile when he was smiling, he
put his arm round her and drew her burning lips to his. "Bedtime," he
said, gaily, "for we've got to be up early in the morning."
Over about them as they clung together the spirit of Millstead, like
a watchful friend, came suddenly close and intimate, and to Speed,
opening his soul to it joyously, it appeared in the likeness of a
golden-haired child, shy yet sombrely passionate—a wraith of a child
that was just like Helen. Above all, they loved each other, these two,
with a love that surrounded and enveloped all things in a magic
haze: they were the perfect lovers. And over them the real corporeal
Millstead brooded in constant magnificent calm.

II

Soon he was swallowed up in the joyous routine of term-time. He

had never imagined that a housemaster had such a large amount of
work to do. There were no early-morning forms during the winter
term, however, and as also it was a housemaster's privilege to
breakfast in his own rooms, Speed began the day with a happy
three-quarters of an hour of newspaper-scanning, envelope-tearing,
and chatting with Helen. After breakfast work began in earnest.
Before term had lasted a week he discovered that he had at least
twice as many duties as in the preceding term; the Head was
certainly not intending to let him slack. There was the drawing and
music of the whole school to superintend, as well as the choir and
chapel-services which, as the once-famous Raggs became more
and more decrepit, fell into Speed's direction almost automatically.
Then also there were a large number of miscellaneous supervisory
duties which the housemasters shared between them, and one or
two, at least, which tradition decreed should be performed entirely by
the junior housemaster. The result of it all was that Speed was, if he
had been in the mood to desire a statutory eight hours' day,
considerably overworked.
It was fortunate that the work was what he loved. He plunged into
it with terrific zest. Lavery's was a large House, and Lavery himself
had judged all its institutions by the test of whether or not they
conduced to an economy in work for him. The result was an
institution that managed itself with rough-and-ready efficiency, that
offered no glaring scandal to the instrusive eye, yet was, in truth,
honeycombed with corruption of a mild sort, and completely under
the sway of powerfully vested interests. Against this and these
Speed set himself out to do final battle. A prudent housemaster, and
certainly one who valued his own personal comfort, would have
postponed the contest, at any rate, until he had become settled in his
position. But Speed, emboldened by the extraordinary success of his
first term, and lured by his own dreams of a Lavery's that should be
the great House at Millstead, would not delay. In his first week he
found five of the prefects enjoying a pleasant little smoking-party in
the Senior prefect's study. They explained to him that Lavery had
never objected to their smoking, provided they did it
unostentatiously, and that Lavery never dreamed of "barging in upon
them" during their evening study-hours. Speed, stung by their slightly
insolent bearing, barked at them in his characteristic staccato voice
when annoyed: "It doesn't matter to me a bit what Lavery used to let
you do. You've got to obey me now, not Lavery. Prefects must set
the example to the others. I shall ask for an undertaking from all of
you that you don't smoke again during term-time. I'll give you till to-
morrow night to decide. Those who refuse will be degraded from
prefecture."
"You can't degrade without the Head's authority," said Smallwood,
the most insolent of the party.
Speed replied, colouring suddenly (for he realised that Smallwood
had spoken the truth): "I know my own business, thank you,
Smallwood."
During the following twenty-four hours four out of the half-dozen
House-prefects gave the required undertaking. The other two,
Smallwood and a fellow named Biffin, refused, "on principle," as they
said, without explaining what exactly the qualification meant. Speed
went promptly to the Head and appealed for authority to degrade
them. He found that they had already poured their tale into the
Head's receptive ears, and that they had given the Head the
impression that he (Speed) in a tactless excess of reforming zeal,
had been listening at keyholes and prying around the study-doors at
night. The Head, after listening to Speed's indignant protest, replied,
suavely: "I think, Mr. Speed"—(Speed's relationship as son-in-law
never tempted either of them to any intimacy of address)—"I think
you must—um, yes—make some allowance for the—um—the
natural inclination of elder boys to—um—to be jealous of privileges.
Smoking is, of course, an—um, yes—an offence against school
rules, but Mr. Lavery was perhaps—um, yes, perhaps—wise in
turning the—um—the blind eye, when the offender was near the top
of the school and where the offence was not flagrant. You must
remember, Mr. Speed, that Smallwood is eighteen years of age, not
so very many years younger than you are yourself. Besides, he is—
um, yes, I think so—captain of the First Fifteen, is he not?—and I—
um—I assure you—his degradation through you would do you an—
um—an incalculable amount of harm in the school. Don't make
yourself unpopular, Mr. Speed. I will send a note round the school,
prefects—um, yes—included, drawing—um—attention to the school
rule against smoking. And I will talk to Smallwood and the other boy
—Biffin, isn't he?—um, yes—privately. Privately, you see—a quiet
friendly conversation in—um—in private, can achieve wonders."
Speed felt that he was being ever so gently snubbed. He left the
Head's study in a state of subdued fury, and his temper was not
improved when Helen seemed rather thoughtlessly inclined to take
Smallwood's side. "Don't get people into trouble, Kenneth," she
pleaded. "I don't think you ought to complain to father about them.
After all, it isn't frightfully wicked to smoke, is it? and I know they all
used to do it in Lavery's time. Why, I've seen them many a time
when I've passed the study-windows in the evenings."
He stared at her for a few seconds, half indignantly, half
incredulously: then, as if on sudden impulse, he smiled and placed
his hands on her shoulders and looked searchingly into her eyes.
"Soft-hearted little kid!" he exclaimed, laughing a slightly forced
laugh. "All the same, I don't think you quite understand my position,
dear."
"Tell me about it then," she said.
Perhaps instinct forewarned him that if he went into details, either
his indignation would break its bounds or else she would make some
further casual and infuriating comment. From both possibilities he
shrank nervously. He said, with an affectation of nonchalance: "Oh,
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com