Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently Mezzetta - Download the ebook with all fully detailed chapters
Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently Mezzetta - Download the ebook with all fully detailed chapters
https://ebookmass.com/product/data-fabric-and-data-mesh-approaches-
with-ai-1st-edition-eberhard-hechler/
https://ebookmass.com/product/data-driven-solutions-to-transportation-
problems-yinhai-wang/
https://ebookmass.com/product/principles-of-data-science-sinan-
ozdemir/
https://ebookmass.com/product/data-driven-harnessing-data-and-ai-to-
reinvent-customer-engagement-1st-edition-tom-chavez/
Data Universe: Organizational Insights with Python:
Embracing Data Driven Decision Making Van Der Post
https://ebookmass.com/product/data-universe-organizational-insights-
with-python-embracing-data-driven-decision-making-van-der-post/
https://ebookmass.com/product/python-for-finance-mastering-data-
driven-finance-2nd-edition/
https://ebookmass.com/product/data-driven-seo-with-python-solve-seo-
challenges-with-data-science-using-python-1st-edition-andreas-
voniatis/
https://ebookmass.com/product/intelligent-data-analysis-from-data-
gathering-to-data-comprehension-deepak-gupta/
https://ebookmass.com/product/modern-data-architecture-on-azure-
design-data-centric-solutions-on-microsoft-azure-1st-edition-sagar-
lad/
Principles of Data Fabric
Sonia Mezzetta
BIRMINGHAM—MUMBAI
Principles of Data Fabric
Copyright © 2023 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, without the prior written permission of the publisher, except in the case
of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express
or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable
for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.
ISBN 978-1-80461-522-5
www.packtpub.com
To my daughter, Melania; you are and will always be my forever inspiration in everything I do. You
are my hero. I know it wasn’t easy at times not having my undivided attention, so thank you for your
patience while I wrote this book.
To Mike, thank you for your love and significant support throughout this journey. I couldn’t have done
this without your help.
To my parents and sisters, thank you for always being there for me and for rooting me on. Family is
everything.
To my loving pets at home, Cody and Stella, and to those pets no longer with us, Bella and Lobo. I
miss you both dearly.
– Sonia Mezzetta
Contributors
Jo Ramos is a distinguished engineer and the Director and Chief Solutions Architect for Data Fabric
at IBM Expert Labs. Jo leads the technology and platform architecture team to support clients on their
data modernization and transformation journey to accelerate the adoption of data and AI enterprise
capabilities. Jo has extensive experience working as a technologist and thought leader across multiple
industries, designing innovative data and analytics solutions for enterprises. His specialties includes Data
Fabric, Data Mesh, DataOps, Data Governance, Data Integration, Big Data, Data Science and Analytics.
Rosalind Radcliffe is an IBM Fellow and CIO DevSecOps CTO. Rosalind is responsible for driving
DevSecOps and application modernization transformation for the IBM CIO office with the goal of
making the office the showcase for hybrid cloud. In this role, she works with the CIO office and partners
on research and development to drive the adoption of common practices and tools. Ultimately, this
effort will transform, standardize, and automate the processes, tools, and methodologies used to make
IBM the most secure, agile, efficient, and automated hybrid cloud engineering organization. In her prior
role, she was responsible for bringing open modern toolchains to the z/OS platform and working with
clients on their DevOps transformation. She is a frequent speaker at conferences, a master inventor, a
member of the IBM Academy of Technology, and the author of Enterprise Bug Busting.
Table of Contents
Preface xiii
2
Show Me the Business Value 19
Digital transformation 19 Trusting your decisions with governed
data23
Data monetization 20
Creating a unified view of your data
Revenue20
with intelligent Data Integration 26
Cost savings 21
Gaining a competitive advantage with
Data Fabric’s value proposition 22 Self-Service26
viii Table of Contents
Summary43
4
Introducing DataOps 45
What is DataOps? 45 Data Fabric with DataOps 57
DataOps’ principles 46 Develop58
The evolution of DataOps 47 Orchestrate58
DataOps’ dimensions 49 Test58
MLOps and AIOps depend on DataOps 52 Deploy59
Monitor59
DataOps’ value 53
From traditional Data Quality to data Summary59
observability54
Table of Contents ix
5
Building a Data Strategy 61
Why create a data strategy? 61 Creating a data strategy document 67
A data maturity framework 62
Data strategy implementation 68
A data maturity assessment 64
Summary70
Creating a data strategy 65
Topics in a data strategy document 66
7
Designing Data Governance 87
Data Governance architecture 87 Operational models 99
Metadata-driven architecture 88
The Data Fabric’s governance
EDA89
applied99
Metadata as a service 90 The Create phase 100
Metadata collection 90 The Ingest phase 102
Metadata integration 92 The Integrate phase 103
Metadata-based events 96 The Consume phase 104
The Archive and Destroy phase 105
The Data Governance layer 97
Active metadata 97 Summary105
Life cycle governance 98
Visit https://ebookmass.com today to explore
a vast collection of ebooks across various
genres, available in popular formats like
PDF, EPUB, and MOBI, fully compatible with
all devices. Enjoy a seamless reading
experience and effortlessly download high-
quality materials in just a few simple steps.
Plus, don’t miss out on exciting offers that
let you access a wealth of knowledge at the
best prices!
x Table of Contents
8
Designing Data Integration and Self-Service 107
DataOps-based architecture 108 Phase 1 – Create phase in the Data
Integration layer 114
Data Integration layer 109
Phases 2 and 3 – Ingest and Integrate
Data management 109
phases in the Data Integration layer 115
Development workflow 111
Phase 4 – Consume phase in the
Self-Service layer 111 Self-Service layer 118
Phase 5 – Archive and Destroy phase 119
Data democratization 112
Data consumption 113 Data Fabric reference
architecture120
Data journey in a Data Fabric
architecture113 Data Fabric architecture highlights 120
Summary122
9
Realizing a Data Fabric Technical Architecture 123
Technical Data Fabric Data Mesh multi-plane
architecture124 requirements132
Data Fabric tools 124 Multi-plane architecture 132
Vendor and open source tools 129 Data Mesh assumptions 135
Summary141
10
Industry Best Practices 143
Top 16 best practices 143 Best practice 3 146
Data strategy best practices 144 Best practice 4 146
Note
The views expressed in the book belong to the author and do not necessarily represent the
opinions or views of their employer, IBM.
• Executive leaders such as chief data officers, chief technology officers, chief information officers,
and data leaders prioritizing strategic investments to execute an enterprise data strategy
• Enterprise architects, data architects, Data Governance roles such as data security, data privacy
roles, and technical leaders tasked with designing and implementing a mature and governed
Self-Service data platform
• Business analysts and data scientists looking to understand their role as data producers or data
consumers in a Self-Service ecosystem leveraging Data Fabric architecture
• Developers such as data engineers, software engineers, and business intelligence developers
looking to comprehend Data Fabric architecture to learn how it achieves the rapid development
of governed, trusted data
This chapter closes with a view on how Data Fabric and Data Mesh can be used together to achieve
rapid data access, high-quality data, and automated Data Governance.
Chapter 4, Introducing DataOps, introduces the DataOps framework. It discusses the business value it
provides and describes the 18 driving principles that make up DataOps. The role of data observability
and its relationship to the Data Quality and Data Governance pillar is explained. This chapter concludes
by explaining how to apply DataOps as an operational model for Data Fabric architecture.
Chapter 5, Building a Data Strategy, kicks off the creation and implementation of a data strategy
document. It describes a data strategy document as a visionary statement and a plan for profitable
revenue and cost savings. You will familiarize yourself with the different sections that should be defined
in a data strategy document, and have a reference of three data maturity frameworks to use as input
in a data strategy. The chapter ends with tips on how Data Fabric architecture can be positioned as
part of a data strategy document.
Chapter 6, Designing a Data Fabric Architecture, sets the foundation for the design of a Data Fabric
architecture. It introduces key architecture concepts and architecture principles that compose the logical
data architecture of a Data Fabric. The three architecture layers, Data Governance, Data Integration, and
Self-Service, in a Data Fabric architecture are introduced. The objectives of each layer are highlighted,
with a discussion on the necessary capabilities represented as components.
Chapter 7, Designing Data Governance, dives into the design of the Data Governance layer of a Data
Fabric architecture. Key architecture patterns, such as metadata-driven and event-driven architectures,
are discussed. The architecture components, such as active metadata, metadata knowledge graphs, and
life cycle governance, are explained. The chapter ends with an explanation of how the Data Governance
layer executes and governs data at each phase in its life cycle.
Chapter 8, Designing Data Integration and Self-Service, drills into the design of the two remaining
architecture layers in a Data Fabric, Data Integration and Self-Service. The Data Integration layer is
reviewed, which focuses on the development of data with a DataOps lens. The Self-Service layer is
also discussed, including how it aims to democratize data. An understanding is provided of how both
architecture layers work with each other, and how they rely on the Data Governance layer. At the end
of the chapter, a Data Fabric reference architecture is presented.
Chapter 9, Realizing a Data Fabric Technical Architecture, positions a technical Data Fabric architecture
as modular and composable, consisting of several tools and technologies. The required capabilities and
the kinds of tools to implement each of the three layers in a Data Fabric architecture are discussed. Two
use cases are reviewed – distributed data management via Data Mesh and regulatory compliance – as
examples of how to apply a Data Fabric architecture. The chapter ends by presenting a Data Fabric
with Data Mesh technical reference architecture.
Chapter 10, Industry Best Practices, presents 16 best practices in data management. Best practices are
grouped into four categories: Data Strategy, Data Architecture, Data Integration and Self-Service, and
Data Governance. Each best practice is described and has a why should you care statement.
xvi Preface
Conventions used
There are a number of text conventions used throughout this book.
Bold: Indicates a new term, an important word, or words that you see onscreen. For instance,
words in menus or dialog boxes appear in bold. Here is an example: “Select System info from the
Administration panel.”
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at customercare@
packtpub.com and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen.
If you have found a mistake in this book, we would be grateful if you would report this to us. Please
visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would
be grateful if you would provide us with the location address or website name. Please contact us at
copyright@packt.com with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you
are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Preface xvii
https://packt.link/free-ebook/9781804615225
Data Fabric architecture, alongside its distinguishing qualities and business value proposition, needs
to first be defined to enable its adoption as part of any data management strategy.
The first part of this book introduces Data Fabric architecture by establishing the core building blocks
and their business value proposition. I offer a different perspective on what defines Data Fabric
architecture than ones on the market today. Data Fabric is a flexible and composable architecture
capable of adopting several data management styles and operational models. Foundational Data
Governance pillars and their intended focus are explained, and a list of key characteristics of what
does and doesn’t define Data Fabric is provided.
By the end of Part 1, you will have an understanding of what Data Fabric architecture is, its
differentiating characteristics and architecture principles, and the impact of not having a Data
Governance-centric architecture.
This part comprises the following chapters:
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookmass.com