Programming Persistent Memory: A Comprehensive Guide For Developers 1st Edition Steve Scargall instant download
Programming Persistent Memory: A Comprehensive Guide For Developers 1st Edition Steve Scargall instant download
https://textbookfull.com/product/programming-persistent-memory-a-
comprehensive-guide-for-developers-1st-edition-steve-scargall/
https://textbookfull.com/product/camping-washington-a-
comprehensive-guide-to-public-tent-and-rv-campgrounds-3rd-
edition-steve-giordano/
https://textbookfull.com/product/scala-programming-a-
comprehensive-beginner-s-guide-to-scala-2nd-edition-claudia-
alves/
https://textbookfull.com/product/the-economics-of-education-a-
comprehensive-overview-2nd-edition-steve-bradley-editor/
https://textbookfull.com/product/quick-boot-a-guide-for-embedded-
firmware-developers-2nd-edition-pete-dice/
The Rust Programming Language 1st Edition Steve Klabnik
https://textbookfull.com/product/the-rust-programming-
language-1st-edition-steve-klabnik/
https://textbookfull.com/product/building-generative-ai-powered-
apps-a-hands-on-guide-for-developers-1st-edition-kansal/
https://textbookfull.com/product/reservoir-modelling-a-practical-
guide-1st-edition-steve-cannon/
https://textbookfull.com/product/linux-for-developers-jumpstart-
your-linux-programming-skills-william-bo-rothwell/
https://textbookfull.com/product/executive-functioning-a-
comprehensive-guide-for-clinical-practice-1st-edition-yana-suchy/
Programming
Persistent
Memory
A Comprehensive Guide for Developers
—
Steve Scargall
Programming Persistent
Memory
A Comprehensive Guide for
Developers
Steve Scargall
Programming Persistent Memory: A Comprehensive Guide for Developers
Steve Scargall
Santa Clara, CA, USA
iii
Table of Contents
iv
Table of Contents
Persistent Libraries��������������������������������������������������������������������������������������������������������������������� 67
libpmem�������������������������������������������������������������������������������������������������������������������������������� 67
libpmemobj���������������������������������������������������������������������������������������������������������������������������� 68
libpmemobj-cpp�������������������������������������������������������������������������������������������������������������������� 68
libpmemkv����������������������������������������������������������������������������������������������������������������������������� 69
libpmemlog���������������������������������������������������������������������������������������������������������������������������� 69
libpmemblk���������������������������������������������������������������������������������������������������������������������������� 69
Tools and Command Utilities������������������������������������������������������������������������������������������������������� 70
pmempool������������������������������������������������������������������������������������������������������������������������������ 70
pmemcheck��������������������������������������������������������������������������������������������������������������������������� 70
pmreorder������������������������������������������������������������������������������������������������������������������������������ 71
Summary������������������������������������������������������������������������������������������������������������������������������������ 71
v
Table of Contents
vi
Table of Contents
vii
Table of Contents
viii
Table of Contents
ix
Table of Contents
Chapter 16: PMDK Internals: Important Algorithms and Data Structures������������ 313
A Pool of Persistent Memory: High-Level Architecture Overview��������������������������������������������� 313
The Uncertainty of Memory Mapping: Persistent Memory Object Identifier����������������������������� 315
Persistent Thread Local Storage: Using Lanes�������������������������������������������������������������������������� 318
Ensuring Power-Fail Atomicity: Redo and Undo Logging���������������������������������������������������������� 320
Transaction Redo Logging��������������������������������������������������������������������������������������������������� 320
Transaction Undo Logging��������������������������������������������������������������������������������������������������� 321
libpmemobj Unified Logging������������������������������������������������������������������������������������������������ 322
Persistent Allocations: The Interface of a Transactional Persistent Allocator���������������������������� 323
Persistent Memory Heap Management: Allocator Design for Persistent Memory�������������������� 324
ACID Transactions: Efficient Low-Level Persistent Transactions����������������������������������������������� 328
Lazy Reinitialization of Variables: Storing the Volatile State on Persistent Memory����������������� 330
Summary���������������������������������������������������������������������������������������������������������������������������������� 331
x
Table of Contents
xi
Table of Contents
Appendix B: How to Install the Persistent Memory Development Kit (PMDK)������ 395
PMDK Prerequisites������������������������������������������������������������������������������������������������������������������ 395
Installing PMDK Using the Linux Distribution Package Repository������������������������������������������� 395
Package Naming Convention����������������������������������������������������������������������������������������������� 396
Searching for Packages Within a Package Repository�������������������������������������������������������� 396
Installing PMDK Libraries from the Package Repository����������������������������������������������������� 398
Installing PMDK on Microsoft Windows������������������������������������������������������������������������������������ 402
Glossary���������������������������������������������������������������������������������������������������������������� 425
Index��������������������������������������������������������������������������������������������������������������������� 429
xii
About the Author
Steve Scargall is a persistent memory software and cloud architect at Intel
Corporation. As a technology evangelist, he supports the enabling and development
effort to integrate persistent memory technology into software stacks, applications,
and hardware architectures. This includes working with independent software
vendors (ISVs) on both proprietary and open source development, original equipment
manufacturers (OEMs), and cloud service providers (CSPs).
Steve holds a Bachelor of Science in computer science and cybernetics from the
University of Reading, UK, where he studied neural networks, AI, and robotics. He
has over 19 years’ experience providing performance analysis on x86 architecture and
SPARC for Solaris Kernel, ZFS, and UFS file system. He performed DTrace debugging in
enterprise and cloud environments during his tenures at Sun Microsystems and Oracle.
xiii
About the Technical Reviewer
Andy Rudoff is a principal engineer at Intel Corporation, focusing on non-volatile
memory programming. He is a contributor to the SNIA NVM Programming Technical
Work Group. His more than 30 years’ industry experience includes design and
development work in operating systems, file systems, networking, and fault management
at companies large and small, including Sun Microsystems and VMware. Andy has
taught various operating systems classes over the years and is a coauthor of the popular
UNIX Network Programming textbook.
xv
About the Contributors
Piotr Balcer is a software engineer at Intel Corporation with many years’ experience
working on storage-related technologies. He holds a Bachelor of Science in engineering
from the Gdańsk University of Technology, Poland, where he studied system software
engineering. Piotr has been working on the software ecosystem for next-generation
persistent memory since 2014.
Eduardo Berrocal joined Intel Corporation as a cloud software engineer in 2017 after
receiving his PhD in computer science from the Illinois Institute of Technology. His
doctoral research focused on data analytics and fault tolerance for high-performance
computing. Past experience includes working as an intern at Bell Labs (Nokia), a research
aid at Argonne National Laboratory, a scientific programmer and web developer at the
University of Chicago, and an intern in the CESVIMA laboratory in Spain.
xvii
About the Contributors
Chet Douglas is a principal software engineer at Intel Corporation and focuses on cloud
software architecture along with operating system and OEM enabling of non-volatile
memory technologies. He has over 14 years’ experience working on various enterprise
and client programs and 28 years of total storage experience. Chet has worked in all
aspects of storage, including storage controller hardware design, SCSI disk/tape/CD
writer firmware architecture, storage management software architecture, Microsoft
Windows* and Linux kernel-mode drivers, enterprise hardware RAID, and client/
workstation software RAID. He holds seven storage-related hardware and software
patents and has a dual Bachelor of Science in electrical engineering and computer
engineering from Clarkson University, New York.
Ken Gibson is the director of persistent memory software architecture within Intel
Corporation’s Data Center Group. Since 2012, Ken and his team have been working with
Intel’s server and software partners to create the open persistent memory programming
model.
Kishor Kharbas is a software engineer on the Java runtime engineering team at Intel
Corporation. For the past eight years, he has been working to optimize Oracle’s OpenJDK
on Intel platforms. This involves Java garbage collection and compiler back-end
optimization.
xviii
About the Contributors
Nicholas Moulin is a cloud software architect at Intel Corporation. Since joining Intel
in 2012, he has focused on enabling and developing persistent memory software for
operating systems and platform firmware and managing persistent memory hardware.
Nicholas is currently working with industry partners to define and improve RAS features
relevant to the persistent memory programming model.
Vineet Singh is a memory and storage tools software engineer at Intel Corporation.
He develops techniques to help developers adapt to the latest memory technologies.
Vineet holds a PhD in philosophy from the University of California and has a Bachelor
of Technology degree from the Indian Institute of Information Technology, Design, and
Manufacturing in Jabalpur.
xix
About the Contributors
Usha Upadhyayula has been with Intel Corporation for 20 years serving in many
different roles. Usha holds a master’s degree in computer science from the University
of South Carolina, and she spent the first few years at Intel developing user-level
applications in C and C++. She later moved to customer-enabling roles for Intel media
processors and support for Intel RAID software. Usha is currently part of the Data Center
Group where she is focused on enabling cloud service providers to fully utilize and
accelerate the adoption of Intel persistent memory products.
xx
Acknowledgments
First and foremost, I would like to thank Ken Gibson for masterminding this book idea
and for gifting me the pleasure of writing and managing it. Your support, guidance, and
contributions have been instrumental in delivering a high-quality product.
If the Vulcan mind-meld or The Matrix Headjack were possible, I could have cloned
Andy Rudoff’s mind and allowed him to work on his daily activities. Instead, Andy’s
infinite knowledge of persistent memory had to be tapped through good old verbal
communication and e-mail. I sincerely thank you for devoting so much time to me and
this project. The results read for themselves.
Debbie Graham was instrumental in helping me manage this colossal project. Her
dedication and support helped drive the project to an on-time completion.
To my friends and colleagues at Intel who contributed content, supported
discussions, helped with decision-making, and reviewed drafts during the book-writing
process. These are the real heroes. Without your heavily invested time and support, this
book would have taken considerably longer to complete. It is a much better product as a
result of the collaborative effort. A huge thanks to all of you.
I'd like to express my sincerest gratitude and appreciation to the people at Apress,
without whom this book could not have been published. From the initial contact and
outline discussions through the entire publishing process to this final polished product,
the Apress team delivered continuous support and assistance. Many thanks to Susan,
Jessica, and Rita. It was a real pleasure working with you.
xxi
Preface
About This Book
Persistent memory is often referred to as non-volatile memory (NVM) or storage
class memory (SCM). In this book, we purposefully use persistent memory as an all-
encompassing term to represent all the current and future memory technologies that
fall under this umbrella. This book introduces the persistent memory technology and
provides answers to key questions. For software developers, those questions include:
What is persistent memory? How do I use it? What APIs and libraries are available?
What benefits can it provide for my application? What new programming methods do I
need to learn? How do I design applications to use persistent memory? Where can I find
information, documentation, and help?
System and cloud architects will be provided with answers to questions such as:
What is persistent memory? How does it work? How is it different than DRAM or SSD/
NVMe storage devices? What are the hardware and operating system requirements?
What applications need or could benefit from persistent memory? Can my existing
applications use persistent memory without being modified?
Persistent memory is not a plug-and-play technology for software applications.
Although it may look and feel like traditional DRAM memory, applications need to be
modified to fully utilize the persistence feature of persistent memory. That is not to say
that applications cannot run unmodified on systems with persistent memory installed,
they can, but they will not see the full potential of what persistent memory offers without
code modification.
Thankfully, server and operating system vendors collaborated very early in the
design phase and already have products available on the market. Linux and Microsoft
Windows already provide native support for persistent memory technologies. Many
popular virtualization technologies also support persistent memory.
For ISVs and the developer community at large, the journey is just beginning. Some
software has already been modified and is available on the market. However, it will
take time for the enterprise and cloud computing industries to adopt and make the
hardware available to the general marketplace. ISVs and software developers need time
to understand what changes to existing applications are required and implement them.
xxiii
Preface
To make the required development work easier, Intel developed and open sourced
the Persistent Memory Development Kit (PMDK) available from https://pmem.io/
pmdk/. We introduce the PMDK in more detail in Chapter 5 and walk through most of
the available libraries in subsequent chapters. Each chapter provides an in-depth guide
so developers can understand what library or libraries to use. PMDK is a set of open
source libraries and tools based on the Storage Networking Industry Association (SNIA)
NVM programming model designed and implemented by over 50 industry partners. The
latest NVM programming model document can be found at https://www.snia.org/
tech_activities/standards/curr_standards/npm. The model describes how software
can utilize persistent memory features and enables designers to develop APIs that take
advantage of NVM features and performance.
Available for both Linux and Windows, PMDK facilitates persistent memory
programming adoption with higher-level language support. C and C++ support is fully
validated. Support for other languages such as Java and Python is work in progress
at the time this book was written. Other languages are expected to also adopt the
programming model and provide native persistent memory APIs for developers. The
PMDK development team welcomes and encourages new contributions to core code,
new language bindings, or new storage engines for the persistent memory key-value
store called pmemkv.
This book assumes no prior knowledge of persistent memory hardware devices
or software development. The book layout allows you to freely navigate the content in
the order you want. It is not required to read all chapters in order, though we do build
upon concepts and knowledge described in previous chapters. In such cases, we make
backward and forward references to relevant chapters and sections so you can learn or
refresh your memory.
B
ook Structure
This book has 19 chapters, each one focusing on a different topic. The book has three
main sections. Chapters 1-4 provide an introduction to persistent memory architecture,
hardware, and operating system support. Chapters 5-16 allow developers to understand
the PMDK libraries and how to use them in applications. Finally, Chapters 17-19 provide
information on advanced topics such as RAS and replication of data using RDMA.
xxiv
Preface
xxv
Preface
xxvi
Preface
The Appendixes have separate procedures for installing the PMDK and utilities
required for managing persistent memory. We also included an update for Java and the
future of the RDMA protocols. All of this content is considered temporal, so we did not
want to include it in the main body of the book.
Intended Audience
This book has been written for experienced application developers in mind. We
intend the content to be useful to a wider readership such as system administrators
and architects, students, lecturers, and academic research fellows to name but a few.
System designers, kernel developers, and anyone with a vested or passing interest in this
emerging technology will find something useful within this book.
Every reader will learn what persistent memory is, how it works, and how operating
systems and applications can utilize it. Provisioning and managing persistent memory
are vendor specific, so we include some resources in the Appendix sections to avoid
overcomplicating the main chapter content.
Application developers will learn, by example, how to integrate persistent memory
in to existing or new applications. We use examples extensively throughout this book
using a variety of libraries available within the Persistent Memory Development Kit
(PMDK). Example code is provided in a variety of programming languages such as C,
C++, JavaScript, and others. We want developers to feel comfortable using these libraries
in their own projects. The book provides extensive links to resources where you can find
help and information.
System administrators and architects of Cloud, high-performance computing,
and enterprise environments can use most of the content of this book to
understand persistent memory features and benefits to support applications and
developers. Imagine being able to deploy more virtual machines per physical server or
provide applications with this new memory/storage tier such that they can keep more
data closer to the CPU or restart in a fraction of the time they could before while keeping
a warm cache of data.
Students, lecturers, and academic research fellows will also benefit from many
chapters within this book. Computer science classes can learn about the hardware,
operating system features, and programming techniques. Lecturers are free use the
content in student classes or to form the basis of research projects such as new persistent
memory file systems, algorithms, or caching implementations.
xxvii
Preface
We introduce tools that profile the server and applications to better understand CPU,
memory, and disk IO access patterns. Using this knowledge, we show how applications
can be modified to take full advantage of persistence using the Persistent Memory
Development Kit (PMDK).
A Future Reference
The book content has been written to provide value for many years. Industry
specification such as ACPI, UEFI, and the SNIA non-volatile programming model will,
unless otherwise stated by the specification, remain backward compatible as new
versions are released. If new form factors are introduced, the approach to programming
remains the same. We do not limit ourselves to one specific persistent memory vendor
or implementation. In places where it is necessary to describe vendor-specific features
or implementations, we specifically call this out as it may change between vendors or
between product generations. We encourage you to read the vendor documentation for
the persistent memory product to learn more.
Developers using the Persistent Memory Development Kit (PMDK) will retain a stable
API interface. PMDK will deliver new features and performance improvements with each
major release. It will evolve with new persistent memory products, CPU instructions,
platform designs, industry specifications, and operating system feature support.
xxviii
Preface
The code examples provided with this book have been tested and validated using
Intel Optane DC persistent memory. Since the PMDK is vendor neutral, they will also
work on NVDIMM-N devices. PMDK will support any future persistent memory product
that enters the market.
The code examples used throughout this book are current at the time of
publication. All code examples have been validated and tested to ensure they compile
and execute without error. For brevity, some of the examples in this book use assert()
statements to indicate unexpected errors. Any production code would likely replace
these with the appropriate error handling actions which would include friendlier
error messages and appropriate error recovery actions. Additionally, some of the code
examples use different mount points to represent persistent memory aware file systems,
for example “/daxfs”, “/pmemfs”, and “/mnt/pmemfs”. This demonstrates persistent
memory file systems can be mounted and named appropriately for the application, just
like regular block-based file systems. Source code is from the repository that accompanies
this book – https://github.com/Apress/programming-persistent-memory.
Since this is a rapidly evolving technology, the software and APIs references
throughout this book may change over time. While every effort is made to be backward
compatible, sometimes software must evolve and invalidate previous versions. For this
reason, it is therefore expected that some of the code samples may not compile on newer
hardware or operating systems and may need to be changed accordingly.
B
ook Conventions
This book uses several conventions to draw your attention to specific pieces of
information. The convention used depends on the type of information displayed.
C
omputer Commands
Commands, programming library, and API function references may be presented in line
with the paragraph text using a monospaced font. For example:
To illustrate how persistent memory is used, let’s start with a sample program
demonstrating the key-value store provided by a library called libpmemkv.
xxix
Preface
S
ource Code
Source code examples taken from the accompanying GitHub repository are shown with
relevant line numbers in a monospaced font. Below each code listing is a reference to
the line number or line number range with a brief description. Code comments use
language native styling. Most languages use the same syntax. Single line comments
will use // and block/multiline comments should use /*..*/. An example is shown in
Listing 1.
xxx
Preface
• Line 45: Here we define a small helper routine, kvprint(), which prints
a key-value pair when called.
Notes
We use a standard format for notes, cautions, and tips when we want to direct your
attention to an important point, for example.
xxxi
Discovering Diverse Content Through
Random Scribd Documents
lawfully engaged in commerce on the high seas, and in waters
of the United States:
“And Whereas, An Executive Proclamation has already been
issued, requiring the persons engaged in these disorderly
proceedings to desist therefrom, calling out a militia force for
the purpose of repressing the same, and convening Congress in
extraordinary session to deliberate and determine thereon:
“Now, therefore, I, Abraham Lincoln, President of the United
States, with a view to the same purposes before mentioned,
and to the protection of the public peace, and the lives and
property of quiet and orderly citizens pursuing their lawful
occupations, until Congress shall have assembled and
deliberated on the said unlawful proceedings, or until the same
shall have ceased, have further deemed it advisable to set on
foot a blockade of the ports within the States aforesaid, in
pursuance of the laws of the United States, and of the laws of
nations in such cases provided. For this purpose a competent
force will be posted so as to prevent entrance and exit of
vessels from the ports aforesaid. If, therefore, with a view to
violate such blockade, a vessel shall approach, or shall attempt
to leave any of the said ports, she will be duly warned by the
commander of one of the blockading vessels, who will indorse
on her register the fact and date of such warning; and if the
same vessel shall again attempt to enter or leave the blockaded
port, she will be captured and sent to the nearest convenient
port, for such proceedings against her and her cargo as prize,
as may be deemed advisable.
“And I hereby proclaim and declare, that if any person,
under the pretended authority of said States, or under any other
pretence, shall molest a vessel of the United States, or the
persons or cargo on board of her, such person will be held
amenable to the laws of the United States for the prevention
and punishment of piracy.
“By the President: Abraham Lincoln.
“William H. Seward, Secretary of State.”
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
textbookfull.com