Skip to content

[pull] master from Snailclimb:master #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pull merged 3 commits into from
Oct 18, 2021
+11 −1
Merged

[pull] master from Snailclimb:master #220

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f014952
feat:补充kafka不重复消费的解答
machitaoX Oct 17, 2021
3d92eca
Merge pull request #1431 from machitaoX/feature_completeKafka
Snailclimb Oct 18, 2021
83519ff
Update Kafka常见面试题总结.md
Snailclimb Oct 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion docs/system-design/distributed-system/message-queue/Kafka常见面试题总结.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,17 @@ acks 的默认值即为1，代表我们的消息被leader副本接收之后就

### Kafka 如何保证消息不重复消费

代办...
**kafka出现消息重复消费的原因：**

- 服务端侧已经消费的数据没有成功提交 offset（根本原因）。
- Kafka 侧 由于服务端处理业务时间长或者网络链接等等原因让 Kafka 认为服务假死，触发了分区 rebalance。

**解决方案：**

- 消费消息服务做幂等校验，比如 Redis 的set、MySQL 的主键等天然的幂等功能。这种方法最有效。
- 将 **`enable.auto.commit`** 参数设置为 false，关闭自动提交，开发者在代码中手动提交 offset。那么这里会有个问题：**什么时候提交offset合适？**
* 处理完消息再提交：依旧有消息重复消费的风险，和自动提交一样
* 拉取到消息即提交：会有消息丢失的风险。允许消息延时的场景，一般会采用这种方式。然后，通过定时任务在业务不繁忙（比如凌晨）的时候做数据兜底。

### Reference

Expand Down