Instant Download gRPC Microservices in Go (MEAP V08) Hüseyin Babal PDF All Chapters
Instant Download gRPC Microservices in Go (MEAP V08) Hüseyin Babal PDF All Chapters
com
https://ebookmeta.com/product/grpc-microservices-in-go-
meap-v08-huseyin-babal/
OR CLICK HERE
DOWLOAD NOW
https://ebookmeta.com/product/shipping-go-develop-deliver-discuss-
design-and-go-again-meap-v08-joel-holmes/
ebookmeta.com
https://ebookmeta.com/product/advent-of-go-microservices-tit-petric/
ebookmeta.com
https://ebookmeta.com/product/go-h-ck-yourself-1st-edition-bryson-
payne/
ebookmeta.com
https://ebookmeta.com/product/the-engaged-scholar-expanding-the-
impact-of-acedemic-research-in-today-s-world-1st-edition-andrew-j-
hoffman/
ebookmeta.com
Decadence A Very Short Introduction David Weir
https://ebookmeta.com/product/decadence-a-very-short-introduction-
david-weir/
ebookmeta.com
https://ebookmeta.com/product/1000-questions-1000-answers-b2-2020-
clean-2020th-edition-nemethne-hock-ildiko/
ebookmeta.com
https://ebookmeta.com/product/bulk-metallic-glasses-and-their-
composites-2nd-edition-muhammad-musaddique-ali-rafique/
ebookmeta.com
https://ebookmeta.com/product/high-stakes-cowboy-west-
protection-0-5-1st-edition-em-petrova/
ebookmeta.com
Are We Really Better Together An Evangelical Perspective
on the Division in The UMC Walter Fenton Rob Renfroe
https://ebookmeta.com/product/are-we-really-better-together-an-
evangelical-perspective-on-the-division-in-the-umc-walter-fenton-rob-
renfroe/
ebookmeta.com
gRPC Microservices in Go MEAP V08
1. Copyright_2023_Manning_Publications
2. welcome
3. 1_Introduction_to_Go_gRPC_Microservices
4. 2_gRPC_Meets_with_Microservices
5. 3_Getting_up_and_running_with_gRPC_and_Golang
6. 4_Microservices_Project_Setup
7. 5_Inter-service_Communication
8. 6_Resilient_Communication
9. 7_Testing_Microservices
10. 8_Deployment
11. 9_Observability
MEAP Edition Manning Early Access Program gRPC Microservices in Go
Version 8
manning.com
welcome
Thank you for purchasing the MEAP Edition of gRPC Microservice in Go.
In this book, we deep dive into the theory of Microservice Architecture and
how it is different from Monolithic Architecture. We also use gRPC and Go
to communicate between services with a simple message and rpc definitions.
Besides step by step implementation of each service, you can also see CI/CD
examples applied to Microservice development. Code examples are enriched
with illustrative diagrams to show you the big picture.
We hope you enjoy the book while reading theoretical information with
detailed examples and applying code samples to implement a real
eCommerce application with Microservice Architecture.
We encourage you to ask any question or provide feedback about live book
content in the Livebook Discussion Forum.
–Hüseyin Babal
In this book
Good architecture design and proper technology selection help you a lot to
build a high-quality product by eliminating repetitive work and providing the
best toolkit for software development and maintenance. While one of the
main advantages of Microservice Architecture is that it can be implemented
in any language, Go is particularly suited for building high-performance
Cloud-Native distributed applications like Microservices in Kubernetes at a
big scale. Microservices with gRPC communication have already enabled
many companies to implement their products with small services based on
their business capabilities and let those services communicate smoothly with
each other and the public. With the help of Go, the distribution of those
services becomes easier due to its fast compilation, ability to generate
executable binaries, and many other reasons that we will see in detail with
real-life examples in the upcoming chapters.
The main goal of this book is to provide production-grade best practices for
gRPC Microservices so that by the end of this book, you will have the self-
confidence to implement the entire system on your own.
1.1.1 Performance
gRPC provides better performance and security than other protocols like
REST with JSON or XML communication as it uses Protocol Buffers, and
HTTP/2 over TLS is straightforward. Protocol Buffers, also known as
Protobuf, is a language and platform-neutral mechanism for serializing
structural data, which you will see in detail in Chapter 3. This mechanism
empowers gRPC to quickly serialize messages into small and compact
messages on both the server and client sides. In the same way, HTTP/2
enables the performance with server-side push, multiplexing, and header
compression, which we will see in more detail in Chapter 5.
gRPC tools and libraries are compatible to work with multiple platforms and
languages, including Go, Java, Python, Ruby, Javascript, C#, and more. The
Protobuf binary wire format, data format as it travels on a wire like in a
network, and well-designed code generation for almost all platforms enables
developers to build performance-critical applications while keeping cross-
platform support. We will see the details of why Protobuf performs well in
inter-service communication in Chapter 3.
Besides business objects, you can define service methods and generate
implementations similarly. Those service functions can be called after you
initialize the gRPC client on the consumer side; again, this client is generated
out of the box.
If an operation is not a good fit for the idempotency use case, you must
provide proper validation errors in a response message that helps you know
when to stop the retry operation. Once you guarantee this idempotency or
proper validation, it is just a definition of retry policy on the gRPC side. Fault
tolerance also focuses on topics like Rate Limiting, Circuit Breaker, and Fault
Injection, which we will see in greater detail in Chapter 6.
1.1.4 Security
In most systems, you may need a security layer to protect your product
against unverified sources. gRPC encourages HTTP/2 over SSL/TLS to
authenticate and encrypt exchanged data between client and server. More
specifically, you can easily set that authentication system up by using
SSL/TLS, ALTS (Application Layer Transport Security), or Token Based
Authentication System, which we will cover in more detail in Chapter 6.
1.1.5 Streaming
Sometimes you may need to divide response data into several chunks to
provide them to the user in a paginated way to reduce bandwidth and return
them to the user quickly. Moreover, if they are only interested in specific
pages, it is not meaningful to return all the data simultaneously. In gRPC,
besides the pagination, you can also stream this data to the consumer instead
of forcing the user to do pagination to get data iteratively. Streaming
shouldn’t necessarily be on the server side; it can also be on the client side or
both sides simultaneously, which is called bi-directional streaming. In a
typical streaming use case, you open the connection once, and the data will
be streamed through this opened connection. You will see different kinds of
usages of streaming use cases, particularly in Chapter 5, while we implement
a complete application within this book.
Since gRPC has built-in HTTP 2.0 support, you can also use Unary and Bi-
directional streaming between clients and servers, resulting in high-speed
communication. With default settings of REST services, multiple client-
server communications can introduce a delay to your overall system
performance.
There are also cases where REST is more beneficial than gRPC. For example,
REST protocol is supported in all kinds of browsers. Since gRPC support is
minimal, you may need to use a proxy layer like gRPC Web
(https://github.com/grpc/grpc-web) to communicate easily with the gRPC
server.
gRPC has lots of advantages, like being able to define messages to exchange
data between services easily. However, regarding readability, JSON and
XML usage in REST has advantages, like changing it freely if there is no
explicit business validation for the changed fields. In contrast, you need to
follow some rules in gRPC to make a change. We will explain this in Chapter
5 in detail.
gRPC has a built-in client and server stub generation mechanism where you
need to use a framework in REST like Swagger Codegen to generate client-
side models. This becomes critical, especially once you have multiple
services and maintain multiple SDKs simultaneously for your customers.
Now that we understand the differences between REST and gRPC let's look
at when it would make sense to use gRPC."
gRPC may not be the proper selection for very simple applications like
startup projects that contain only 1-2 services since maintaining the proto
files that contain service definitions is not easy, especially for inexperienced
users.
Figure 1.1 Architecture diagram of an e-commerce product built with Go Microservices on top of
Kubernetes, including CI/CD flow and Observability
1.4.1 Microservices
Microservice projects are full of challenges, especially at the beginning of the
project, and you will often hear the following questions in your architectural
decision meetings.
Each service could have different behavior, like different resource requests,
different scale factors, different language runtime, and so on. Again, they are
just configurations within Kubernetes deployments that can be appropriately
configured for each service. For example, Product service needs more
capacity or scaling factor than other services since most customers search and
view products during the day. You don’t need to scale all the services
simultaneously in Kubernetes as you do for a monolithic application. It can
be handled by adding scaling factors and resource capacity to specific
services.
The main output for each service will be a cloud-native application, which
means you can deploy this service to any other container runtime like AWS
Fargate, AWS ECS, even docker for local development, and so on with a
little modification.
You can easily use gRPC tools to generate stubs on your local environment,
but wouldn’t it be better to generate them whenever some changes are pushed
to the remote repository? Also, you can generate artifacts to deploy them to
an experimental or stable environment after merging them into the main
branch. Modern VCS providers like Github, Gitlab, and Bitbucket already
have that kind of integration, so there is not much custom implementation
needed for this level of automation.
A green check after CI/CD job execution does not mean everything is fine;
there should be a correct level of check mechanism for the services. For
example, good coverage of unit tests, proper integration tests to check 3rd
party integrations like MySQL, Kubernetes, or AWS, contract testing for
service-to-service communication, static code analysis, and vulnerability
checks are the parts for a good start to have a reliable codebase in the main
branch.
Dashboards, panels, and graphs for your system provide a good start for
better Observability. Still, we should focus on introducing new metrics and
creating specific alarms based on them to be notified when you are away
from your dashboards. As an example, Prometheus (https://prometheus.io),
an open-source event monitoring and alerting tool, can be used to collect
system and application metrics, and there can be new alert configurations
based on those metrics like, “notify once the memory usage percentage > 80
for a specific service”. Logs are also good sources for insights since you can
calculate error rates in real-time. You can even create alert configurations
based on log patterns within modern log management tools like Elastic Stack
(Elasticsearch, Logstash, and other Elastic integration products).
With a good monitoring setup, there will not only be good insight for service-
to-service communication but there will also be insights for service-to-third-
party integrations. For example, it will be possible to detect performance
problems between service and database or service to third-party API that is
out of the organization’s control.
API Gateways are widely used to prevent the above scenarios by following
certain principles, like setting up a proper authentication/authorization system
quickly, introducing rate-limiting to restrict users for their request capacity,
etc. If you are using Kubernetes already, you can handle them with built-in
features like adding authorization and rate limiting configuration to nginx
controllers; otherwise, you have other options like using API Gateway
products.
We just finished a chapter with a few in-depth topics, but it sets the context
for the big picture to help you decide if Go gRPC Microservices is a good fit
for your use case. We will cover each section in detail in upcoming chapters.
1.5 Summary
gRPC performs well in inter-service communications because it uses
binary serialization for the data and transfers it through the HTTP/2
protocol.
gRPC allows you to do client streaming, server streaming, and bi-
directional streaming that allows you to send multiple requests or
receive multiple responses in parallel.
Stable Client–Server interaction in gRPC Microservices is easy because
of automatic code generation.
REST is popular primarily due to its broader browser support, but you
can still use a gRPC web proxy (e.g., https://github.com/grpc/grpc-web)
for REST-to-gRPC conversion.
Due to its high portability, Go is one of the best languages for cloud-
native applications, like microservices in kubernetes.
Using HTTP/2 over SSL/TLS end-to-end encryption connection in
gRPC eliminates most of the security concerns for a Microservice
2 gRPC Meets with Microservices
This chapter covers
Comparing advantages and disadvantages of Microservice Architecture
to Monolithic Architecture
Understanding communication patterns in Microservice Architecture
Analyzing service discovery mechanisms
Addressing the reasons behind how Go and gRPC boosts reliable inter-
service communication as well as development productivity
2.1.1 Development
All the modern IDEs are designed to support monolithic applications. For
example, you can open a multi-module maven project in IntelliJ IDEA
(https://www.jetbrains.com/idea/) or create a modular Go project with Goland
(https://www.jetbrains.com/go/) that you can easily open and navigate within
the codebase.
However, problems can arise as your codebase grows. For example, suppose
you have tens of modules within a monolithic application and try to open
them simultaneously. In that case, it is possible to overload the IDE, which
negatively impacts productivity, and it may not be relevant to open them all if
you don't need some of them.
Additionally, if you have no proper isolation on your test cases, you may run
all the tests whenever you make a small change in your codebase. The bigger
codebase, the longer compile and testing time.
2.1.2 Deployment
Deploying a monolithic application means copying a standalone package or
folder hierarchy to the server or a container runtime. However, monoliths
may be an obstacle to frequent deployments in Continuous Deployment since
it is hard to deploy and test in a reasonable time interval. You need to deploy
the entire application, even if you introduce a tiny change to a specific
component. For example, let's say that we introduced a small change in the
newsletter component responsible for serving the newsletter, and now we
want to test and deploy it to production. To have a green flag for tests, we
need to run all the tests, even though we haven't changed anything in other
components like Payment, Order, etc. In the same way, we need to build the
system to generate one artifact even though the changes are only in the
newsletter component.
2.1.3 Scaling
Let's say that you have a monolithic application that needs 16G of memory,
and the most critical module out of 10 modules is customer service. When
you scale this monolithic application by 2, you will end up with 32G of
memory allocation. Let's say that the custom module needs 2G memory to
run efficiently. Wouldn't it be better to only scale the customer module by
two that need an extra 2G of memory instead of 16G?
Now that we understand the pros and cons of Monolithic Architecture and
scalability models, let's take a look at how Microservices Architecture is a
form of Y-Axis Scaling
First, you must decide whether microservice architecture fits your product
architecture well. As we already addressed, starting with monolithic
architecture is a best practice that allows you to understand your business
capabilities. Once you start having scalability problems, less productive
development, or longer release lifecycles, you can assess your environment to
see whether functional decomposition is a good fit for your application. Once
you decide to use microservice architecture, you might have independently
scalable services, small projects that contain only specific context during
development, and faster deployments due to faster test verification and small
release artifacts.
Let's assume you are familiar with your business model and know how to
decompose your application into small services. You will have other
challenges not visible in monolithic applications, such as handling data
consistency and inter-service communication.
Figure 2.2 Consistency of the Order data is managed by transaction boundaries which are
committed and rollback
A typical transaction can be expressed with the steps begin and
commit/rollback, where you begin a transaction and execute the actual
operation; then, it may end up committing data into the data store or roll back
the entire operation made in the transaction. Now that we understand data
consistency can be easily handled in Monolithic Architecture let's look at
how it is handled in Microservice Architecture.
As you can see, since transaction steps are spanned across the services, it is
not something that you can handle with an annotation or two lines of code.
However, there are widely used practices with Saga, so you don't need to
reinvent the wheel from scratch for your use cases. Choreography and
Orchestrator-based Saga are the most popular patterns widely used for inter-
service communication to have consistent data.
Now let's look at how to apply one of the above notations to a real-life use-
case for order creation flow.
In a typical order creation flow, Saga is created in Order Service, and Order
is created with PENDING state. It sends an event called order_created right
after Order is persisted and consumed by Payment Service. Payment Service
will try to charge the customer and send another event, payment_created or
payment_failed. If it fails, Order Service will be notified, and Order will be
marked as FAILED. Shipping Service will consume the event and initiate the
shipping process if it is successful. Finally, it will create another event for
failure or success which will cause Order status to be marked as FAILED or
SUCCESS.
'NICHOLAS.
'By the grace of God, we, Nicholas I, Emperor and Autocrat of all the
Russias, King of Poland, &c., &c.
'In our constant solicitude for the happiness of the nations which
Providence has confided to our government, we are occupied in
fixing the basis for the future organization of the kingdom of Poland,
having regard to the true interests and positions of the country, and
to the local wants and manners of the inhabitants.
'GENERAL DISPOSITIONS.
'Art. 1. The kingdom of Poland is forever to be re-united to the
Russian empire, and form an inseparable part of that empire. It shall
have a particular administration conformably to its local necessities,
as well as a civil and military code. The statutes and the laws of
cities and towns remain in full vigor.
'Art. 2. The Crown of the kingdom of Poland is hereditary in our
person and in our heirs and successors, agreeably to the order of
succession to the throne prescribed by all the Russias.
'Art. 3. The Coronation of the Emperors of all the Russias and Kings
of Poland shall be one and the same ceremonial, which shall take
place at Moscow, in the presence of a deputation from the kingdom
of Poland, which shall assist at that solemnity with the deputies from
the other parts of the empire.
'Art. 4. In the possible event of a regency in Russia, the power of the
regent or regentess of the empire will extend over the kingdom of
Poland.
'Art. 5. The freedom of worship is guarantied; every one is at liberty
to exercise his religion openly, under the protection of Government;
and the difference of Christian faiths shall never prove a pretext for
the violation of the rights and privileges which are allowed to all the
inhabitants. The Roman Catholic religion, being that of the majority
of our Polish subjects, shall be the object of especial protection of
the Government.
'Art. 6. The funds which the Roman Catholic clergy possess, and
those of the Greek church united, shall be considered as the
common and inviolable property of the hierarchy of each of those
creeds.
'Art. 7. The protection of the laws is assured to all the inhabitants
without distinction of rank or class. Each shall be empowered to
assume dignities or to exercise public functions, according to his
personal merits or talents.
'Art. 8. Individual liberty is guarantied and protected by the existing
laws. No one shall be deprived of his liberty, or called to justice, if he
be not a transgressor of the law in all the forms prescribed. Every
one detained shall be apprised of the motive of arrest.
'Art. 9. Each person arrested must submit to a delay of three days to
be heard and judged of, according to the forms of law, before
competent tribunals: if he be found innocent, he will instantly obtain
his liberty. He will be equally restored to liberty who shall furnish a
sufficient surety.
'Art. 10. The form of judicial inquests directed against the superior
functionaries of the kingdom, and against persons accused of high
treason, shall be determined by a particular law, the foundation of
which shall be accordant with the other laws of our empire.
'Art. 11. The right of property of individuals, and of corporations, is
declared sacred and inviolable, inasmuch as it will be conformable to
the existing laws. All the subjects of the kingdom of Poland are
perfectly free to quit the country, and to carry away their goods,
provided they conform to the regulations published to that effect.
'Art. 12. The penalty of confiscation shall not be enforced but against
state crimes of the first class, as may be hereafter determined by
particular laws.
'Art. 13. Publication of sentiments, by means of the press, shall be
subjected to restrictions which will protect religion, the inviolability of
superior authority, the interests of morals, and personal
considerations. Particular regulations, to this effect, will be published
according to the principles which serve as a basis to this object in
the other parts of our empire.
'Art. 14. The kingdom of Poland shall proportionably contribute to
the general expenditure and to the wants of the empire. The
proportion of taxes will be stated hereafter.
'Art. 15. All contributions and all taxes which existed in November,
1830, shall be levied after the manner formerly settled till the new
fixing of taxes.
'Art. 16. The treasury of the kingdom of Poland, and all the other
branches of the administration, shall be separated from the
administration of the other parts of the kingdom.
'Art. 17. The public debt of Poland, acknowledged by us, shall be
guarantied as formerly, by the government, and indemnified by the
receipts of the kingdom.
'Art. 18. The bank of the kingdom of Poland, and the laws respecting
credit, shall continue under the protection of Government.
'Art. 19. The mode of commercial transactions between the Russian
empire and the kingdom of Poland shall be regulated according to
the respective interests of the two countries.
'Art. 20. Our army in the empire and in the kingdom shall compose
one in common, without distinction of Russian or Polish troops. We
shall reserve to ourselves a future decision of this, by an especial
law, by what arrangement, and upon what basis, the kingdom of
Poland shall participate with our army. The number of troops which
shall serve as the military defence of the kingdom will be also
ultimately determined upon by a law.
'Art. 21. Those of our subjects of the empire of Russia, who are
established in the kingdom of Poland, who possess or shall possess,
real property in that country, shall enjoy all the rights of natives. It
shall be the same with those of our subjects of the kingdom of
Poland, who shall establish themselves, and shall possess property,
in the other provinces of the empire. We reserve to ourselves to
grant hereafter letters of naturalization to other persons, as well to
strangers as to Russians, who are not yet established there. Those
of our subjects of the Russian empire who may reside for a certain
time in Poland, and those of our subjects of the kingdom of Poland
who may sojourn in the other parts of the empire, are subject to the
laws of the country where they reside.
'Art. 22. The superior administration of the kingdom of Poland is
confided to a council of administration, which shall govern the
kingdom in our name, under the presidency of the governor of the
kingdom.
'Art. 23. The council of administration is composed of the governor
of the kingdom, of superior directors, who superintend the
commissions, and among whom are divided the interests of the
administration, of comptroller, presiding over the supreme Chamber
of Finance, and of other members, whom we shall appoint by special
orders.'
FOOTNOTES:
[85] Not having a copy of this address in the original, we make
use of a rather unsatisfactory translation, which we find in the
journals of the day.
r s t u w x y z.
1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside
the United States, check the laws of your country in addition to
the terms of this agreement before downloading, copying,
displaying, performing, distributing or creating derivative works
based on this work or any other Project Gutenberg™ work. The
Foundation makes no representations concerning the copyright
status of any work in any country other than the United States.
1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if
you provide access to or distribute copies of a Project
Gutenberg™ work in a format other than “Plain Vanilla ASCII” or
other format used in the official version posted on the official
Project Gutenberg™ website (www.gutenberg.org), you must,
at no additional cost, fee or expense to the user, provide a copy,
a means of exporting a copy, or a means of obtaining a copy
upon request, of the work in its original “Plain Vanilla ASCII” or
other form. Any alternate format must include the full Project
Gutenberg™ License as specified in paragraph 1.E.1.
• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
about donations to the Project Gutenberg Literary Archive
Foundation.”
• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.
1.F.